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Preface for the Instructor 


You are about to teach a course that will probably give students their second 
exposure to linear algebra. During their first brush with the subject, your 
students probably worked with Euclidean spaces and matrices. In contrast, 
this course will emphasize abstract vector spaces and linear maps. 

The audacious title of this book deserves an explanation. Almost all 
linear algebra books use determinants to prove that every linear operator on 
a finite-dimensional complex vector space has an eigenvalue. Determinants 
are difficult, nonintuitive, and often defined without motivation. To prove the 
theorem about existence of eigenvalues on complex vector spaces, most books 
must define determinants, prove that a linear map is not invertible if and only 
if its determinant equals 0, and then define the characteristic polynomial. This 
tortuous (torturous?) path gives students little feeling for why eigenvalues 
exist. 

In contrast, the simple determinant-free proofs presented here (for example, 
see 5.21) offer more insight. Once determinants have been banished to the 
end of the book, a new route opens to the main goal of linear algebra — 
understanding the structure of linear operators. 

This book starts at the beginning of the subject, with no prerequisites 
other than the usual demand for suitable mathematical maturity. Even if your 
students have already seen some of the material in the first few chapters, they 
may be unaccustomed to working exercises of the type presented here, most 
of which require an understanding of proofs. 

Here is a chapter-by-chapter summary of the highlights of the book: 

• Chapter 1 : Vector spaces are defined in this chapter, and their basic proper- 
ties are developed. 

• Chapter 2: Linear independence, span, basis, and dimension are defined in 
this chapter, which presents the basic theory of finite-dimensional vector 
spaces. 
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• Chapter 3: Linear maps are introduced in this chapter. The key result here 
is the Fundamental Theorem of Linear Maps (3.22): if T is a linear map 
on V, then dim V — dim null T + dim range T. Quotient spaces and duality 
are topics in this chapter at a higher level of abstraction than other parts 
of the book; these topics can be skipped without running into problems 
elsewhere in the book. 

• Chapter 4: The part of the theory of polynomials that will be needed 
to understand linear operators is presented in this chapter. This chapter 
contains no linear algebra. It can be covered quickly, especially if your 
students are already familiar with these results. 

• Chapter 5: The idea of studying a linear operator by restricting it to small 
subspaces leads to eigenvectors in the early part of this chapter. The 
highlight of this chapter is a simple proof that on complex vector spaces, 
eigenvalues always exist. This result is then used to show that each linear 
operator on a complex vector space has an upper-triangular matrix with 
respect to some basis. All this is done without defining determinants or 
characteristic polynomials ! 

• Chapter 6: Inner product spaces are defined in this chapter, and their basic 
properties are developed along with standard tools such as orthonormal 
bases and the Gram-Schmidt Procedure. This chapter also shows how 
orthogonal projections can be used to solve certain minimization problems. 

• Chapter 7 : The Spectral Theorem, which characterizes the linear operators 
for which there exists an orthonormal basis consisting of eigenvectors, 
is the highlight of this chapter. The work in earlier chapters pays off 
here with especially simple proofs. This chapter also deals with positive 
operators, isometries, the Polar Decomposition, and the Singular Value 
Decomposition. 

• Chapter 8: Minimal polynomials, characteristic polynomials, and gener- 
alized eigenvectors are introduced in this chapter. The main achievement 
of this chapter is the description of a linear operator on a complex vector 
space in terms of its generalized eigenvectors. This description enables 
one to prove many of the results usually proved using Jordan Form. For 
example, these tools are used to prove that every invertible linear operator 
on a complex vector space has a square root. The chapter concludes with a 
proof that every linear operator on a complex vector space can be put into 
Jordan Form. 
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• Chapter 9: Linear operators on real vector spaces occupy center stage in 
this chapter. Here the main technique is complexification, which is a natural 
extension of an operator on a real vector space to an operator on a complex 
vector space. Complexification allows our results about complex vector 
spaces to be transferred easily to real vector spaces. For example, this 
technique is used to show that every linear operator on a real vector space 
has an invariant subspace of dimension 1 or 2. As another example, we 
show that that every linear operator on an odd-dimensional real vector space 
has an eigenvalue. 

• Chapter 10: The trace and determinant (on complex vector spaces) are 
defined in this chapter as the sum of the eigenvalues and the product of the 
eigenvalues, both counting multiplicity. These easy-to-remember defini- 
tions would not be possible with the traditional approach to eigenvalues, 
because the traditional method uses determinants to prove that sufficient 
eigenvalues exist. The standard theorems about determinants now become 
much clearer. The Polar Decomposition and the Real Spectral Theorem are 
used to derive the change of variables formula for multivariable integrals in 
a fashion that makes the appearance of the determinant there seem natural. 

This book usually develops linear algebra simultaneously for real and 
complex vector spaces by letting F denote either the real or the complex 
numbers. If you and your students prefer to think of F as an arbitrary field, 
then see the comments at the end of Section l.A. I prefer avoiding arbitrary 
fields at this level because they introduce extra abstraction without leading 
to any new linear algebra. Also, students are more comfortable thinking 
of polynomials as functions instead of the more formal objects needed for 
polynomials with coefficients in finite fields. Finally, even if the beginning 
part of the theory were developed with arbitrary fields, inner product spaces 
would push consideration back to just real and complex vector spaces. 

You probably cannot cover everything in this book in one semester. Going 
through the first eight chapters is a good goal for a one-semester course. If 
you must reach Chapter 10, then consider covering Chapters 4 and 9 in fifteen 
minutes each, as well as skipping the material on quotient spaces and duality 
in Chapter 3. 

A goal more important than teaching any particular theorem is to develop in 
students the ability to understand and manipulate the objects of linear algebra. 
Mathematics can be learned only by doing. Fortunately, linear algebra has 
many good homework exercises. When teaching this course, during each 
class I usually assign as homework several of the exercises, due the next class. 
Going over the homework might take up a third or even half of a typical class. 
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Major changes from the previous edition: 

• This edition contains 561 exercises, including 337 new exercises that were 
not in the previous edition. Exercises now appear at the end of each section, 
rather than at the end of each chapter. 

• Many new examples have been added to illustrate the key ideas of linear 
algebra. 

• Beautiful new formatting, including the use of color, creates pages with an 
unusually pleasant appearance in both print and electronic versions. As a 
visual aid, definitions are in beige boxes and theorems are in blue boxes (in 
color versions of the book). 

• Each theorem now has a descriptive name. 

• New topics covered in the book include product spaces, quotient spaces, 
and duality. 

• Chapter 9 (Operators on Real Vector Spaces) has been completely rewritten 
to take advantage of simplifications via complexification. This approach 
allows for more streamlined presentations in Chapters 5 and 7 because 
those chapters now focus mostly on complex vector spaces. 

• Elundreds of improvements have been made throughout the book. For 
example, the proof of Jordan Form (Section 8.D) has been simplified. 

Please check the website below for additional information about the book. I 
may occasionally write new sections on additional topics. These new sections 
will be posted on the website. Your suggestions, comments, and corrections 
are most welcome. 

Best wishes for teaching a successful linear algebra class! 

Sheldon Axler 
Mathematics Department 
San Francisco State University 
San Francisco, CA 94132, USA 

website: linear.axler.net 
e-mail: linear@axler.net 
Twitter: SAxlerLinear 


Preface for the Student 


You are probably about to begin your second exposure to linear algebra. Unlike 
your first brush with the subject, which probably emphasized Euclidean spaces 
and matrices, this encounter will focus on abstract vector spaces and linear 
maps. These terms will be defined later, so don’t worry if you do not know 
what they mean. This book starts from the beginning of the subject, assuming 
no knowledge of linear algebra. The key point is that you are about to 
immerse yourself in serious mathematics, with an emphasis on attaining a 
deep understanding of the definitions, theorems, and proofs. 

You cannot read mathematics the way you read a novel. If you zip through a 
page in less than an hour, you are probably going too fast. When you encounter 
the phrase “as you should verify”, you should indeed do the verification, which 
will usually require some writing on your part. When steps are left out, you 
need to supply the missing pieces. You should ponder and internalize each 
definition. For each theorem, you should seek examples to show why each 
hypothesis is necessary. Discussions with other students should help. 

As a visual aid, definitions are in beige boxes and theorems are in blue 
boxes (in color versions of the book). Each theorem has a descriptive name. 

Please check the website below for additional information about the book. I 
may occasionally write new sections on additional topics. These new sections 
will be posted on the website. Your suggestions, comments, and corrections 
are most welcome. 

Best wishes for success and enjoyment in learning linear algebra! 

Sheldon Axler 
Mathematics Department 
San Francisco State University 
San Francisco, CA 94132, USA 

website: linear.axler.net 
e-mail: linear@axler.net 
Twitter: @AxlerLinear 
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CHAPTER 

1 






Rene Descartes explaining his 
work to Queen Christina of 
Sweden. Vector spaces are a 
generalization of the 
description of a plane using 
two coordinates, as published 
by Descartes in 1637. 


Vector Spaces 


Linear algebra is the study of linear maps on finite-dimensional vector spaces. 
Eventually we will learn what all these terms mean. In this chapter we will 
define vector spaces and discuss their elementary properties. 

In linear algebra, better theorems and more insight emerge if complex 
numbers are investigated along with real numbers. Thus we will begin by 
introducing the complex numbers and their basic properties. 

We will generalize the examples of a plane and ordinary space to R” 
and C”, which we then will generalize to the notion of a vector space. The 
elementary properties of a vector space will already seem familiar to you. 

Then our next topic will be subspaces, which play a role for vector spaces 
analogous to the role played by subsets for sets. Finally, we will look at sums 
of subspaces (analogous to unions of subsets) and direct sums of subspaces 
(analogous to unions of disjoint sets). 

LEARNING OBJECTIVES FOR THIS CHAPTER 

■ basic properties of the complex numbers 

■ R” and C n 

■ vector spaces 

■ subspaces 

■ sums and direct sums of subspaces 

© Springer International Publishing 2015 1 
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CHAPTER 1 Vector Spaces 


1.A 


R n and C n 


Complex Numbers 

You should already be familiar with basic properties of the set R of real 
numbers. Complex numbers were invented so that we can take square roots of 
negative numbers. The idea is to assume we have a square root of — 1, denoted 
i, that obeys the usual rules of arithmetic. Here are the formal definitions: 


1.1 Definition complex numbers 

• A complex number is an ordered pair (a. b ), where a, b eR, but 
we will write this as a + bi. 

• The set of all complex numbers is denoted by C : 

C = {a + bi : a,b e R}. 

• Addition and multiplication on C are defined by 

(a + bi) + (c + di) = (a + c) + (b + d)i, 

(a + bi)(c + di) = (ac — bd) + (ad + bc)i: 

here a,b, c,d eR 


If a c R, we identify a + 0/ with the real number a. Thus we can think 
of R as a subset of C. We also usually write 0 + bi as just bi, and we usually 
write 0+1/ as just /. 

Using multiplication as defined 
above, you should verify that i 2 = — 1. 
Do not memorize the formula for the 
product of two complex numbers; you 
can always rederive it by recalling that 
/ 2 = — 1 and then using the usual rules 
of arithmetic (as given by 1.3). 


The symbol i was first used to de- 
note V— 1 by Swiss mathematician 
Leonhard Euler in 1 777. 


1.2 Example Evaluate (2 + 3/)(4 + 5/). 

(2 + 3i)(4 + 50 = 2 • 4 + 2 ■ (5/) + (3/) • 4 + (30(50 
= 8 + 10/ + 12/ - 15 
= -7 + 22/ 


Solution 
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i .3 Properties of complex arithmetic 

commutativity 

a + p = P + a and aP = pa for all a, ft e C; 

associativity 

(a + P) + A = a + (P + A) and (ap)X = a(PX) for all a, p, A e C; 

identities 

A + 0 = A and A1 = A for all A e C; 

additive inverse 

for every a e C, there exists a unique P € C such that a + P = 0; 

multiplicative inverse 

for every a e C with a ^ 0, there exists a unique p e C such that 
aP = 1 ; 


distributive property 

A(a + P) = \a + \p for all A, a, p e C. 

The properties above are proved using the familiar properties of real 
numbers and the definitions of complex addition and multiplication. The 
next example shows how commutativity of complex multiplication is proved. 
Proofs of the other properties above are left as exercises. 


1.4 Example Show that aP — pa for alia, p, A e C. 

Solution Suppose a — a + bi and P — c + di , where a , b,c,d € R. Then 
the definition of multiplication of complex numbers shows that 

aP — (a + bi)(c + di) 

= ( ac — bd) + (ad + bc)i 


and 


pa = (c + di)(a + bi) 

— ( ca — db) + ( cb + da)i. 

The equations above and the commutativity of multiplication and addition of 
real numbers show that aP = Pa. 
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1.5 Definition —a, subtraction, 1 /a, division 
Let eC. 

• Let —a denote the additive inverse of a. Thus —a is the unique 
complex number such that 

a + (—a) = 0. 

• Subtraction on C is defined by 

fi-a. = ft + (-a). 


• For a / 0, let 1 /a denote the multiplicative inverse of a. Thus 1 /a 
is the unique complex number such that 

0f(l/0f) = 1. 

• Division on C is defined by 

P/a = P(l/a). 

So that we can conveniently make definitions and prove theorems that 
apply to both real and complex numbers, we adopt the following notation: 

1.6 Notation F 

Throughout this book, F stands for either R or C. 

Thus if we prove a theorem involving 
F, we will know that it holds when F is 
replaced with R and when F is replaced 
with C. 

Elements of F are called scalars. The word “scalar”, a fancy word for 
“number”, is often used when we want to emphasize that an object is a number, 
as opposed to a vector (vectors will be defined soon). 

For a € F and m a positive integer, we define a m to denote the product of 
a with itself m times: 

tfl 

a = . 

m times 

Clearly ( a m ) n = a mn and (ap) m — a m P' n for all a. p € F and all positive 
integers m,n. 


The letter F is used because R and 
C are examples of what are called 

fields. 
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Lists 

Before defining R" and C", we look at two important examples. 

1.7 Example R 2 andR 3 

• The set R 2 , which you can think of as a plane, is the set of all ordered 
pairs of real numbers: 

R 2 = {(x, y) : x,y € R}. 

• The set R 3 , which you can think of as ordinary space, is the set of all 
ordered triples of real numbers: 

R 3 = {(x,y,z) : x,y,z e R). 

To generalize R 2 and R 3 to higher dimensions, we first need to discuss the 
concept of lists. 

1.8 Definition list, length 

Suppose n is a nonnegative integer. A list of length n is an ordered 
collection of n elements (which might be numbers, other lists, or more 
abstract entities) separated by commas and surrounded by parentheses. A 
list of length n looks like this: 

(x 1 ,...,x n ). 

Two lists are equal if and only if they have the same length and the same 
elements in the same order. 

Thus a list of length 2 is an ordered 
pair, and a list of length 3 is an ordered 
triple. 

Sometimes we will use the word list without specifying its length. Re- 
member, however, that by definition each list has a finite length that is a 
nonnegative integer. Thus an object that looks like 

(xi,x 2 , ■■■), 

which might be said to have infinite length, is not a list. 

A list of length 0 looks l ik e this: ( ). We consider such an object to be a 
list so that some of our theorems will not have trivial exceptions. 

Lists differ from sets in two ways: in lists, order matters and repetitions 
have meaning; in sets, order and repetitions are irrelevant. 


Many mathematicians call a list of 
length n an n-tnple. 
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1.9 Example lists versus sets 

• The lists (3, 5) and (5, 3) are not equal, but the sets {3, 5} and {5, 3} are 
equal. 

• The lists (4, 4) and (4, 4, 4) are not equal (they do not have the same 
length), although the sets {4, 4} and {4, 4, 4} both equal the set {4}. 


To define the higher-dimensional analogues of R 2 and R 3 , we will simply 
replace R with F (which equals R or C) and replace theFana 2 or 3 with an 
arbitrary positive integer. Specifically, fix a positive integer n for the rest of 
this section. 

i.io Definition F” 

F” is the set of all lists of length n of elements of F : 

F" = x„) : Xj e F for j = 1 n}. 

For (xi, . . . , x n ) e F” and j e {1, we say that xj is the j th 
coordinate of (xi x n ). 

If F = R and n equals 2 or 3, then this definition of F” agrees with our 
previous notions of R 2 and R 3 . 

1.11 Example C 4 is the set of all lists of four complex numbers: 

c 4 = {(zi,z 2 ,z 3 ,z 4 ) : zi,z 2 ,Z3,z 4 e C}. 


For an amusing account of how 
R 3 would be perceived by crea- 
tures living in R 2 , read Flatland: 
A Romance of Many Dimensions, 
by Edwin A. Abbott. This novel, 
published in 1884, may help you 
imagine a physical space of four or 
more dimensions. 


If n > 4, we cannot visualize R" 
as a physical object. Similarly, C 1 can 
be thought of as a plane, but for n > 2, 
the human brain cannot provide a full 
image of C”. However, even if n is 
large, we can perform algebraic manip- 
ulations in F” as easily as in R 2 or R 3 . 
For example, addition in F” is defined 
as follows: 


SECTION 1. A R" and C 
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1.12 Definition addition in F” 

Addition in F” is defined by adding corresponding coordinates: 

(xi, . . . ,x„) + (vi, . . ■ ,y n ) = Ot + yi,...,x n + y„). 

Often the mathematics of F” becomes cleaner if we use a single letter to 
denote a list of n numbers, without explicitly writing the coordinates. For 
example, the result below is stated with x and y in F” even though the proof 
requires the more cumbersome notation of (xi x n ) and ( Vi , . . . , y n ). 

1.13 Commutativity of addition in F" 

If x, y € F", then x + y = y + x. 

Proof Suppose x = (xi , . . . , x„) and y = (y i , . . . , y n ). Then 

x f y = (xi x n ) + Oi yn) 

= (xi + y\, ... ,x n + y n ) 

= (yi +x\,...,y n +x n ) 

= (jl> • • • > yn) + (xi,...,X„) 

= y + x, 

where the second and fourth equalities above hold because of the definition of 
addition in F” and the third equality holds because of the usual commutativity 
of addition in F. ■ 

If a single letter is used to denote 
an element of F” , then the same letter 
with appropriate subscripts is often used 
when coordinates must be displayed. For example, if x e F” , then letting x 
equal (xi, . . . , x n ) is good notation, as shown in the proof above. Even better, 
work with just x and avoid explicit coordinates when possible. 

1.14 Definition 0 

Let 0 denote the list of length n whose coordinates are all 0: 


The symbol m means “end of the 
proof”. 


0 = (0 , 0 ). 
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Here we are using the symbol 0 in two different ways — on the left side of the 
equation in 1.14, the symbol 0 denotes a list of length n, whereas on the right 
side, each 0 denotes a number. This potentially confusing practice actually 
causes no problems because the context always makes clear what is intended. 


1.15 Example Consider the statement that 0 is an additive identity for F” : 

x + 0 = x for all x <e F” . 

Is the 0 above the number 0 or the list 0? 

Solution Here 0 is a list, because we have not defined the sum of an element 
of F” (namely, x) and the number 0. 



Elements of R 2 can be 
thought of as points 
or as vectors. 



A vector. 


Mathematical models of the econ- 
omy can have thousands of vari- 
ables, say Xi, . . . , Xsooo, which 
means that we must operate in 
R 5000 . Such a space cannot be 
dealt with geometrically. However, 
the algebraic approach works well. 
Thus our subject is called linear 
algebra. 


A picture can aid our intuition. We 
will draw pictures in R 2 because we 
can sketch this space on 2-dimensional 
surfaces such as paper and blackboards. 
A typical element of R 2 is a point x = 
(xi , JC 2 ). Sometimes we think of x not 
as a point but as an arrow starting at the 
origin and ending at (xi , X2), as shown 
here. When we think of x as an arrow, 
we refer to it as a vector. 

When we think of vectors in R 2 as 
arrows, we can move an arrow parallel 
to itself (not changing its length or di- 
rection) and still think of it as the same 
vector. With that viewpoint, you will 
often gain better understanding by dis- 
pensing with the coordinate axes and 
the explicit coordinates and just think- 
ing of the vector, as shown here. 

Whenever we use pictures in R 2 
or use the somewhat vague language 
of points and vectors, remember that 
these are just aids to our understand- 
ing, not substitutes for the actual math- 
ematics that we will develop. Although 
we cannot draw good pictures in high- 
dimensional spaces, the elements of 
these spaces are as rigorously defined 
as elements of R 2 . 


SECTION 1. A R" and C 


9 


For example, (2,-3, 17, n, V2) is an element of R 5 , and we may casually 
refer to it as a point in R 5 or a vector in R 5 without worrying about whether 
the geometry of R 5 has any physical meaning. 

Recall that we defined the sum of two elements of F” to be the element of 
F” obtained by adding corresponding coordinates; see 1.12. As we will now 
see, addition has a simple geometric interpretation in the special case of R 2 . 

Suppose we have two vectors x and 
y in R 2 that we want to add. Move 
the vector y parallel to itself so that its 
initial point coincides with the end point 
of the vector x, as shown here. The 
sum x T y then equals the vector whose 
initial point equals the initial point of 
x and whose end point equals the end 
point of the vector y, as shown here. 

In the next definition, the 0 on the right side of the displayed equation 
below is the list 0 6 F". 



x + y 


The sum of two vectors. 


1.16 Definition additive inverse in Y n 

For x e F", the additive inverse of x, denoted — x, is the vector — x € F n 
such that 

x + (— x) = 0. 

In other words, if x = (xi ,x n ), then — x = (— X\ , . . . , — x n ). 


For a vector x 6 R 2 , the additive in- 
verse —x is the vector parallel to x and 
with the same length as x but pointing in 
the opposite direction. The figure here 
illustrates this way of thinking about the 
additive inverse in R 2 . 

Having dealt with addition in F" , we 4 vector and its additive inverse ' 
now turn to multiplication. We could 

define a multiplication in F” in a similar fashion, starting with two elements 
of F” and getting another element of F” by multiplying corresponding coor- 
dinates. Experience shows that this definition is not useful for our purposes. 
Another type of multiplication, called scalar multiplication, will be central 
to our subject. Specifically, we need to define what it means to multiply an 
element of F” by an element of F. 
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1.17 Definition scalar multiplication in F” 

The product of a number A and a vector in F” is computed by multiplying 
each coordinate of the vector by A : 

A(xi, . ,.,x n ) = (Axi, . . . , A x n )\ 

here A 6 F and (x\, . . . ,x n ) € F" . 


In scalar multiplication, we multi- 
ply together a scalar and a vector, 
getting a vector. You may be famil- 
iar with the dot product in R 2 or 
R 3 , in which we multiply together 
two vectors and get a scalar. Gen- 
eralizations of the dot product will 
become important when we study 
inner products in Chapter 6. 



/ 

(1/2)*/ 

(-3/2) %j 


Scalar multiplication. 


Scalar multiplication has a nice ge- 
ometric interpretation in R 2 . If A is a 
positive number and x is a vector in 
R 2 , then Ax is the vector that points 
in the same direction as x and whose 
length is A times the length of x. In 
other words, to get Ax, we shrink or 
stretch x by a factor of A, depending on 
whether A < 1 or A > 1 . 

If A is a negative number and x is a 
vector in R 2 , then Ax is the vector that 
points in the direction opposite to that 
of x and whose length is | A | times the 
length of x, as shown here. 


Digression on Fields 

A field is a set containing at least two distinct elements called 0 and 1, along 
with operations of addition and multiplication satisfying all the properties 
listed in 1.3. Thus R and C are fields, as is the set of rational numbers along 
with the usual operations of addition and multiplication. Another example of 
a field is the set {0, 1} with the usual operations of addition and multiplication 
except that 1 + 1 is defined to equal 0. 

In this book we will not need to deal with fields other than R and C. 
However, many of the definitions, theorems, and proofs in linear algebra that 
work for both R and C also work without change for arbitrary fields. If you 
prefer to do so, throughout Chapters 1 , 2, and 3 you can think of F as denoting 
an arbitrary field instead of R or C, except that some of the examples and 
exercises require that for each positive integer n we have 1 + 1 + • • • + 1 ^ 0. 


n times 
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EXERCISES l.A 

1 Suppose a and b are real numbers, not both 0. Find real numbers c and 
d such that 

l/(a + hi) = c + di. 

2 Show that 

— 1 4- \/3i 
2 

is a cube root of 1 (meaning that its cube equals 1). 

3 Find two distinct square roots of i . 

4 Show that a + yd = yd + a for all a, yd € C. 

5 Show that (a + yd) + A = a + (yd + A) for all a, yd, A e C. 

6 Show that (afi ) A = a(j3X) for all a, yd, A e C. 

7 Show that for every a e C, there exists a unique (1 e C such that 
a + yd = 0. 

8 Show that for every a e C with of ^ 0, there exists a unique yd e C such 
that cqd = 1 . 

9 Show that A(a + yd) = Aa + Ayd for all A, a, yd e C. 

10 Find x e R 4 such that 

(4,-3, 1,7) + 2x = (5, 9, -6, 8). 

11 Explain why there does not exist A e C such that 

A (2 - 3/, 5 + 4/, -6 + li) = (12 - 5 i, 7 + 22 i, -32 - 9 /). 

12 Show that {x + y) + z — x + (y + z ) for all x, y, z eF". 

13 Show that ( ab)x = a(bx ) for all x e F” and all a, b e F. 

14 Show that lx = x for all x e F". 

15 Show that X(x + y) = Xx + Ay for all A e F and all x,ye F”. 

16 Show that ( a + b)x = ax + bx for all a, b e F and all x e F”. 
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Definition of Vector Space 

The motivation for the definition of a vector space comes from properties of 
addition and scalar multiplication in F” : Addition is commutative, associative, 
and has an identity. Every element has an additive inverse. Scalar multiplica- 
tion is associative. Scalar multiplication by 1 acts as expected. Addition and 
scalar multiplication are connected by distributive properties. 

We will define a vector space to be a set V with an addition and a scalar 
multiplication on V that satisfy the properties in the paragraph above. 

1.18 Definition addition, scalar multiplication 

• An addition on a set V is a function that assigns an element u+v e V 
to each pair of elements u,v € V. 

• A scalar multiplication on a set V is a function that assigns an ele- 
ment Xv € V to each A e F and each v € V. 

Now we are ready to give the formal definition of a vector space. 

1.19 Definition vector space 

A vector space is a set V along with an addition on V and a scalar multi- 
plication on V such that the following properties hold: 

commutativity 

u + v = v + u for all u , v e V; 

associativity 

(u + v) + w = u + (v + w) and ( ab)v = a(bv) for all u, v, w € V 
and all a , b e F ; 

additive identity 

there exists an element 0 e V such that v + 0 = v for all v e V ; 

additive inverse 

for every v e V, there exists w e V such that v + w = 0; 

multiplicative identity 

lv = v for all v e V; 

distributive properties 

a{u + v) = au + av and (a + b)v = av + bv for all a,b e F and 
all u , v e V. 


1.B 
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The following geometric language sometimes aids our intuition. 

1.20 Definition vector, point 

Elements of a vector space are called vectors or points. 

The scalar multiplication in a vector space depends on F. Thus when we 
need to be precise, we will say that V is a vector space over F instead of 
saying simply that V is a vector space. For example, R" is a vector space over 
R, and C” is a vector space over C. 

1 .21 Definition real vector space, complex vector space 

• A vector space over R is called a real vector space. 

• A vector space over C is called a complex vector space. 

Usually the choice of F is either obvious from the context or irrelevant. 
Thus we often assume that F is lurking in the background without specifically 
mentioning it. 

With the usual operations of addition The simplest vector space contains 
and scalar multiplication, F" is a vector only one point. In other words, {0} 
space over F, as you should verify. The is a vector space. 
example of F” motivated our definition 
of vector space. 


1 .22 Example F°° is defined to be the set of all sequences of elements 
ofF: 

F°° = {(xi, X 2 , . ■ .) : Xj € F for j — 1,2,...}. 

Addition and scalar multiplication on F°° are defined as expected: 

(xi,x 2 ,. ..) + (yi,y 2 , ...) = (xi+ yi,x 2 + yi, ■■■), 

X(X \ , X2, . . . ) = (Axi, XX2, 

With these definitions, F°° becomes a vector space over F, as you should 
verify. The additive identity in this vector space is the sequence of all 0’s. 


The simplest vector space contains 
only one point. In other words, {0} 
is a vector space. 


Our next example of a vector space involves a set of functions. 
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1.23 Notation F 5 

• If S is a set, then F s denotes the set of functions from S to F. 

• For f,gs F 5 , the sum f + g e F 5 is the function defined by 

(/ + £)(*) = f(x) + g(x) 

for all x € S. 

• For 1 e F and / e F 5 , the product X f e F‘ S is the function 
defined by 

(A /)(*) = A f{x) 

for all x € S. 


As an example of the notation above, if S is the interval [0, 1] and F = R, 
then R[0’i] is the set of real- valued functions on the interval [0, 1], 

You should verify all three bullet points in the next example. 


1.24 Example F s is a vector space 

• If S is a nonempty set, then F ,s (with the operations of addition and 
scalar multiplication as defined above) is a vector space over F. 

• The additive identity of F ,s is the function 0 : S — ► F defined by 

0(jc) = 0 

for all x € S. 

• For / e F 5 , the additive inverse of / is the function — / : S — > F 
defined by 

(-/)(*) = -fix) 

for all x € S. 


Our previous examples of vector 
spaces, F” and F°°, are special cases 
of the vector space F 5 because a list of 
length n of numbers in F can be thought 
of as a function from {1, 2, to F 
and a sequence of numbers in F can be 
thought of as a function from the set of 
positive integers to F. In other words, we can think of F" as F^ 1,2 ’- ,w ^ and 
we can think of F°° as F* 1,2 ’-*. 


The elements of the vector space 
R [o,i] 

are real-valued functions on 
[0, 1], not lists. In general, a vector 
space is an abstract entity whose 
elements might be lists, functions, 
or weird objects. 
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Soon we will see further examples of vector spaces, but first we need to 
develop some of the elementary properties of vector spaces. 

The definition of a vector space requires that it have an additive identity. 
The result below states that this identity is unique. 

1.25 Unique additive identity 

A vector space has a unique additive identity. 

Proof Suppose 0 and O' are both additive identities for some vector space V. 
Then 

O' = O' + 0 = 0 + O' = 0, 

where the first equality holds because 0 is an additive identity, the second 
equality comes from commutativity, and the third equality holds because 0' 
is an additive identity. Thus 0' = 0, proving that V has only one additive 
identity. ■ 

Each element v in a vector space has an additive inverse, an element w in 
the vector space such that v + w = 0. The next result shows that each element 
in a vector space has only one additive inverse. 

1.26 Unique additive inverse 

Every element in a vector space has a unique additive inverse. 

Proof Suppose V is a vector space. Let v g V. Suppose w and w' are additive 
inverses of v. Then 

w = w + 0 = w + (v + w') = (w + v) + W = 0 + w' = w . 

Thus w = w', as desired. ■ 

Because additive inverses are unique, the following notation now makes 
sense. 

1.27 Notation -v, w — v 
Let v,w € V. Then 

• —v denotes the additive inverse of v; 

• w — v is defined to be w + (— v). 
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Almost all the results in this book involve some vector space. To avoid 
having to restate frequently that V is a vector space, we now make the 
necessary declaration once and for all: 

1.28 Notation V 

For the rest of the book, V denotes a vector space over F. 

In the next result, 0 denotes a scalar (the number 0 e F) on the left side of 
the equation and a vector (the additive identity of V) on the right side of the 
equation. 

1 .29 The number 0 times a vector 

Ov = 0 for every v € V. 


Proof For v e V, we have 

Ov = (0 + 0)v = Ov + Ov. 

Adding the additive inverse of 0i’ to both 
sides of the equation above gives 0 = 
Ov, as desired. ■ 

In the next result, 0 denotes the addi- 
tive identity of V. Although their proofs 
are similar, 1.29 and 1.30 are not identical. More precisely, 1.29 states that 
the product of the scalar 0 and any vector equals the vector 0, whereas 1 .30 
states that the product of any scalar and the vector 0 equals the vector 0. 

1 .30 A number times the vector 0 
ciO — 0 for every a € F. 

Proof For a e F, we have 

a0 = a( 0 + 0) = aO + aO. 

Adding the additive inverse of a 0 to both sides of the equation above gives 
0 = aO, as desired. ■ 

Now we show that if an element of V is multiplied by the scalar — 1 , then 
the result is the additive inverse of the element of V. 


Note that 1.29 asserts something 
about scalar multiplication and the 
additive identity of V. The only 
part of the definition of a vector 
space that connects scalar multi- 
plication and vector addition is the 
distributive property. Thus the dis- 
tributive property must be used in 
the proof of 1 .29. 
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1.31 The number -1 times a vector 
(— l)v = —v for every v € V. 

Proof For v e V, we have 

v + (-l)v = lv + (-l)v = (1 + ( — l))v = Ov = 0. 

This equation says that (— l)v, when added to v, gives 0. Thus (— l)v is the 
additive inverse of v, as desired. ■ 

EXERCISES l.B 

1 Prove that — (— v) = v for every v € V. 

2 Suppose a e F, v e V, and av — 0. Prove that a — 0 or v = 0. 

3 Suppose v, w € V. Explain why there exists a unique x € V such that 
v + 3x = w. 

4 The empty set is not a vector space. The empty set fails to satisfy only 
one of the requirements listed in 1.19. Which one? 

5 Show that in the definition of a vector space (1.19), the additive inverse 
condition can be replaced with the condition that 

Ov = 0 for all v e V. 

Here the 0 on the left side is the number 0, and the 0 on the right side is 
the additive identity of V. (The phrase “a condition can be replaced” in a 
definition means that the collection of objects satisfying the definition is 
unchanged if the original condition is replaced with the new condition.) 

6 Let oo and — oo denote two distinct objects, neither of which is in R. 
Define an addition and scalar multiplication on R U {oo} U {— oo} as you 
could guess from the notation. Specifically, the sum and product of two 


real numbers 

is as usual, and for / 

e R 

define 




— oo 

if t < 0, 



oo if t 

< 0, 

too = < 

0 

if t = 0, 


-oo) = < 

0 if 

= 0, 


.oo 

if t > 0, 



—oo if t 

> o, 

t + oo = 

oo + t 

= oo. 

t + 

-oo) = 

(-oo) + t 

= -oo, 

00 + 00 = 

oo, 

(-oo) + (- 

-oo) = 

= — oo, 

oo + (- 

oo) = 0. 


Is R U {oo} U {— oo} a vector space over R? Explain. 
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l.C 


Subspaces 


By considering subspaces, we can greatly expand our examples of vector 
spaces. 


1.32 Definition subspace 

A subset U of V is called a subspace of V if U is also a vector space 
(using the same addition and scalar multiplication as on V). 


1.33 Example {(jti,jt 2 , 0) : xi,X 2 e F} is a subspace of F 3 . 


Some mathematicians use the term 
linear subspace, which means the 
same as subspace. 


The next result gives the easiest way 
to check whether a subset of a vector 
space is a subspace. 


i .34 Conditions for a subspace 

A subset U of V is a subspace of V if and only if U satisfies the following 
three conditions: 

additive identity 

0 e U 

closed under addition 

u,w € U implies u + w e U ; 

closed under scalar multiplication 

a € F and u € U implies au € U. 

Proof If U is a subspace of V, then U 
satisfies the three conditions above by 
the definition of vector space. 

Conversely, suppose U satisfies the 
three conditions above. The first con- 
dition above ensures that the additive 
identity of V is in U. 

The second condition above ensures 
that addition makes sense on U. The 
third condition ensures that scalar mul- 
tiplication makes sense on U. 


The additive identity condition 
above could be replaced with the 
condition that U is nonempty ( then 
taking u e U, multiplying it by 0, 
and using the condition that U is 
closed under scalar multiplication 
would imply that 0 € U). However, 
if U is indeed a subspace of V, 
then the easiest way to show that U 
is nonempty is to show that 0 € U. 





SECTION l.C Subspaces 19 


If u e U, then —u [which equals (— l)w by 1.31] is also in U by the third 
condition above. Hence every element of U has an additive inverse in U. 

The other parts of the definition of a vector space, such as associativity 
and commutativity, are automatically satisfied for U because they hold on the 
larger space V. Thus U is a vector space and hence is a subspace of V. m 

The three conditions in the result above usually enable us to determine 
quickly whether a given subset of V is a subspace of V. You should verify all 
the assertions in the next example. 

1.35 Example subspaces 

(a) If b € F, then 

{(x\,X2, x' 3 , A' 4 ) e F 4 : X 3 = 5x4 + b} 
is a subspace of F 4 if and only if b = 0. 

(b) The set of continuous real-valued functions on the interval [0, 1] is a 
subspace of R^ 0,1 !. 

(c) The set of differentiable real- valued functions on R is a subspace of R r . 

(d) The set of differentiable real-valued functions / on the interval (0, 3) 
such that /'( 2) = b is a subspace of R (0,3) if and only if b = 0. 

(e) The set of all sequences of complex numbers with limit 0 is a subspace 
of C°°. 


Verifying some of the items above 
shows the linear structure underlying 
parts of calculus. For example, the sec- 
ond item above requires the result that 
the sum of two continuous functions is 
continuous. As another example, the 
fourth item above requires the result 
that for a constant c, the derivative of 
c f equals c times the derivative of /. 

The subspaces of R 2 are precisely {0}, R 2 , and all lines in R 2 through the 
origin. The subspaces of R 3 are precisely {0}, R 3 , all lines in R 3 through the 
origin, and all planes in R 3 through the origin. To prove that all these objects 
are indeed subspaces is easy — the hard part is to show that they are the only 
subspaces of R 2 and R 3 . That task will be easier after we introduce some 
additional tools in the next chapter. 


Clearly {0} is the smallest sub- 
space of V and V itself is the 
largest subspace of V. The empty 
set is not a subspace of V because 
a subspace must be a vector space 
and hence must contain at least 
one element, namely, an additive 
identity. 
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Sums of Subspaces 

When dealing with vector spaces, we 
are usually interested only in subspaces, 
as opposed to arbitrary subsets. The 
notion of the sum of subspaces will be 
useful. 


The union of subspaces is rarely a 
subspace (see Exercise 12), which 
is why we usually work with sums 
rather than unions. 


1.36 Definition sum of subsets 

Suppose U\, , U m are subsets of V. The sum of U\, ... , U m . denoted 
U\ + • • • + U m , is the set of all possible sums of elements of U \ , . . . , U m . 
More precisely, 

U\ + • • • + U m = {mi + • • • + u m : Mi 6 U\,...,u m € U m }. 

Let’s look at some examples of sums of subspaces. 

1 .37 Example Suppose U is the set of all elements of F 3 whose second 
and third coordinates equal 0, and W is the set of all elements of F 3 whose 
first and third coordinates equal 0: 

U = {(x, 0, 0) e F 3 : x e F} and W = {(0, y, 0) e F 3 : y e F}. 
Then 

U + W = {(x, y, 0) : x,y e F}, 

as you should verify. 


1.38 Example Suppose that U — {(x,x,y,y) e F 4 : i,j e F} and 
W — {(x, x, x, y) e F 4 : x, y e F}. Then 

U + W — {(x, x, y, z) e F 4 : x,y,z e F}, 

as you should verify. 

The next result states that the sum of subspaces is a subspace, and is in 
fact the smallest subspace containing all the summands. 

1.39 Sum of subspaces is the smallest containing subspace 

Suppose Ui, ... , U m are subspaces of V. Then U\ + • • ■ + U m is the 
smallest subspace of V containing U\, ... , U m . 
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Proof It is easy to see that 0 e U\ + • • ■ + U m and that U\ + • • • + U m 
is closed under addition and scalar multiplication. Thus 1.34 implies that 
U\ + • • • + U m is a subspace of V. 


Clearly U\,...,U m are all con- 
tained in U\ + ■ ■ ■ + U m (to see this, 
consider sums u\ + • • • + u m where 
all except one of the m’s are 0). Con- 
versely, every subspace of V contain- 
ing U \ , . . . , U m contains U\ 4 \-U m 

(because subspaces must contain all fi- 
nite sums of their elements). Thus 
U\ + • • • + U m is the smallest subspace 
of V containing U\, , U m . m 


Sums of subspaces in the theory 
of vector spaces are analogous 
to unions of subsets in set theory. 
Given two subspaces of a vector 
space, the smallest subspace con- 
taining them is their sum. Analo- 
gously, given two subsets of a set, 
the smallest subset containing them 
is their union. 


Direct Sums 

Suppose U\, , U m are subspaces of V. Every element of U\ + • • ■ + U m 
can be written in the form 


m i + • • • + w m > 

where each uj is in Uj. We will be especially interested in cases where each 
vector in Ci + • • • + U m can be represented in the form above in only one 
way. This situation is so important that we give it a special name: direct sum. 

1.40 Definition direct sum 
Suppose U \ , . . . , U m are subspaces of V. 

• The sum U\ + • • • + U m is called a direct sum if each element 
of U\ + • • ■ + U m can be written in only one way as a sum 
Mi + • • • + u m , where each uj is in Uj . 

• If Ci + • • • + U m is a direct sum, then Ci © • • • © U m denotes 
Ci + • • ■ + U m , with the © notation serving as an indication that 
this is a direct sum. 


1 .41 Example Suppose C is the subspace of F 3 of those vectors whose 
last coordinate equals 0, and W is the subspace of F 3 of those vectors whose 
first two coordinates equal 0: 

C = {(x,y,0) e F 3 : x,y e F} and W = {(0, 0, z) e F 3 : z e F}. 
Then F 3 = C © W, as you should verify. 


(a In 
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1.42 Example Suppose Uj is the subspace of F” of those vectors whose 
coordinates are all 0, except possibly in the j th slot (thus, for example, 
U2 — {( 0 - x, 0 , . . . , 0 ) e F" : x e F}). Then 

F" = lh ©•••©(/„, 

as you should verify. 

Sometimes nonexamples add to our understanding as much as examples. 


1.43 Example Let 

U r = {(x, y, 0 ) e F 3 : x, y 6 F}, 

U 2 = {( 0 , 0 , z) e F 3 : z e F}, 

U3 = {( 0 , y, y) e F 3 : y e F}. 

Show that U] + U 2 + U3 is not a direct sum. 

Solution Clearly F 3 = U\ + U2 + U3, because every vector (x, y, z) € F 3 
can be written as 

(x, y, z) — (x, y, 0) + (0, 0 , z) + (0, 0 , 0), 

where the first vector on the right side is in U\, the second vector is in U2, 
and the third vector is in U3. 

However, F 3 does not equal the direct sum of U\ . U2. L3, because the 
vector (0, 0, 0) can be written in two different ways as a sum u 1 +1/2 + M3, 
with each Uj in Uj. Specifically, we have 

( 0 , 0 , 0 ) = ( 0 , 1 , 0 ) + ( 0 , 0 , 1 ) + ( 0 ,- 1 , - 1 ) 

and, of course, 

(0, 0, 0) = (0, 0, 0) + (0, 0, 0) + (0, 0, 0), 

where the first vector on the right side of each equation above is in Ui , the 
second vector is in U2, and the third vector is in U3. 


The definition of direct sum requires 
that every vector in the sum have a 
unique representation as an appropriate 
sum. The next result shows that when 
deciding whether a sum of subspaces 
is a direct sum, we need only consider 
whether 0 can be uniquely written as an 
appropriate sum. 


The symbol ©, which is a plus 
sign inside a circle, serves as a re- 
minder that we are dealing with a 
special type of sum of subspaces — 
each element in the direct sum can 
be represented only one way as a 
sum of elements from the specified 
subspaces. 
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1 .44 Condition for a direct sum 

Suppose U \ , . . . , U m are subspaces of V. Then U\ + • • • + U m is a direct 
sum if and only if the only way to write 0 as a sum u \ + • • • + u m , where 
each u j is in Uj, is by taking each uj equal to 0. 

Proof First suppose U i + ■ ■ ■ + U m is a direct sum. Then the definition of 

direct sum implies that the only way to write 0 as a sum u\ + \-u m , where 

each Uj is in Uj , is by taking each Uj equal to 0. 

Now suppose that the only way to write 0 as a sum u\ + ■ ■ ■ + u m , where 

each U j is in Uj, is by taking each Uj equal to 0. To show that U\ H F U m 

is a direct sum, let v e U\ + ■ • • + U m . We can write 

v = Ml + • • • + u m 

for some wi e U\ u m e U m . To show that this representation is unique, 

suppose we also have 

V = Vi H F v m , 

where iq e U\, ... ,v m e U m . Subtracting these two equations, we have 
0 = (Mi - Vi) 4 F (U m ~V m )- 

Because u\ — vi e U\, . . . , u m — v m € U m . the equation above implies that 
each Uj — Vj equals 0. Thus mi — v\. . . . ,u m — v m , as desired. ■ 

The next result gives a simple condition for testing which pairs of sub- 
spaces give a direct sum. 

i .45 Direct sum of two subspaces 

Suppose U and W are subspaces of V. Then U + W is a direct sum if 
and only if U fl W = {0}. 

Proof First suppose that U + W is a direct sum. If v e U fl W, then 
0 = v + (— v), where v € U and — v e IT. By the unique representation 
of 0 as the sum of a vector in U and a vector in W, we have v = 0. Thus 
U n W — {0}, completing the proof in one direction. 

To prove the other direction, now suppose U <1 W = {0}. To prove that 
U + IF is a direct sum, suppose u € U, w € W, and 

0 = u + w. 

To complete the proof, we need only show that u = w = 0 (by 1 .44). The 
equation above implies that u — —w € W. Thus u e U n IF. Hence u — 0, 
which by the equation above implies that w = 0, completing the proof. ■ 
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Sums of subspaces are analogous 
to unions of subsets. Similarly, di- 
rect sums of subspaces are analo- 
gous to disjoint unions of subsets. 
No two subspaces of a vector space 
can be disjoint, because both con- 
tain 0. So disjointness is replaced, 
at least in the case of two sub- 
spaces, with the requirement that 
the intersection equals {0}. 


The result above deals only with 
the case of two subspaces. When ask- 
ing about a possible direct sum with 
more than two subspaces, it is not 
enough to test that each pair of the 
subspaces intersect only at 0. To see 
this, consider Example 1.43. In that 
nonexample of a direct sum, we have 

Ui n u 2 = Ui n u 3 = u 2 n u 3 = {0}. 


EXERCISES l.C 

1 For each of the following subsets of F 3 4 5 6 7 8 , determine whether it is a sub- 
space of F 3 : 

(a) {( xi,x 2 ,x 3 ) e F 3 : xi + 2x 2 + 3x3 = 0}; 

(b) {(xi,X 2 ,x 3 ) e F 3 : xi + 2x 2 + 3x 3 = 4}; 

(c) {(xi,x 2 ,x 3 ) e F 3 : xix 2 x 3 = 0}; 

(d) {(xi,x 2 ,x 3 ) e F 3 : xi = 5x 3 }. 

2 Verify all the assertions in Example 1.35. 

3 Show that the set of differentiable real- valued functions / on the interval 
(—4, 4) such that /'(— 1) = 3/(2) is a subspace of R 

4 Suppose b € R. Show that the set of continuous real-valued functions / 
on the interval [0, 1] such that L / = b is a subspace of r[° 4] if and 
only if b — 0. 

5 Is R 1 2 a subspace of the complex vector space C 2 ? 

6 (a) Is {(a, b, c) e R 3 : a 3 — b 3 } a subspace of R 3 ? 

(b) Is {(a, b, c) e C 3 : a 3 = b 3 } a subspace of C 3 ? 

7 Give an example of a nonempty subset U of R 2 such that U is closed 
under addition and under taking additive inverses (meaning —u € U 
whenever u € U ), but U is not a subspace of R 2 . 

8 Give an example of a nonempty subset U of R 2 such that U is closed 
under scalar multiplication, but U is not a subspace of R 2 . 
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9 A function / : R R is called periodic if there exists a positive number 

p such that f(x) = f(x + p) for all x € R. Is the set of periodic 
functions from R to R a subspace of R r ? Explain. 

10 Suppose U i and U 2 are subspaces of V. Prove that the intersection 
U\ n U 2 is a subspace of V. 

11 Prove that the intersection of every collection of subspaces of V is a 
subspace of V. 

12 Prove that the union of two subspaces of V is a subspace of V if and 
only if one of the subspaces is contained in the other. 

13 Prove that the union of three subspaces of V is a subspace of V if and 
only if one of the subspaces contains the other two. 

[This exercise is surprisingly harder than the previous exercise, possibly 
because this exercise is not true if we replace F with afield containing 
only two elements .] 

14 Verify the assertion in Example 1.38. 

15 Suppose U is a subspace of V. What is U + U ? 

16 Is the operation of addition on the subspaces of V commutative? In other 
words, if U and W are subspaces of V, is U + W — W + U1 

17 Is the operation of addition on the subspaces of V associative? In other 
words, if (/, , U 2 . 1/3 are subspaces of V, is 

(t/i + U 2 ) + U 3 = u 1 + (U 2 + 1 / 3 )? 

18 Does the operation of addition on the subspaces of V have an additive 
identity? Which subspaces have additive inverses? 

19 Prove or give a counterexample: if U 1 , U 2 , W are subspaces of V such 
that 

U\ + W = U 2 + W, 

then U\ = U 2 . 

20 Suppose 

U — {(x,x, y, y) e F 4 : jc, y e F}. 

Find a subspace W of F 4 such that F 4 = U © W. 
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21 Suppose 

U — {(x, y, x + y, x - y, 2x) e F 5 : x, y e F}. 

Find a subspace W of F 5 such that F 5 = U © W. 

22 Suppose 

U — {(x, y, x + y, x — y, 2x) eF 5 :x,je F}. 

Find three subspaces W \ , W 2 , W 3 of F 5 , none of which equals {0}, such 
that F 5 = U © W x © W 2 © W 3 . 

23 Prove or give a counterexample: if U\, U 2 , W are subspaces of V such 
that 

V = Ui © W and V = U 2 © W, 

then U\ = U 2 - 

24 A function / : R —*■ R is called even if 

/(-*) = fix) 

for all x e R. A function / : R — >• R is called odd if 

f(-x) = ~f(x) 

for all x e R. Let U e denote the set of real- valued even functions on R 
and let U a denote the set of real- valued odd functions on R. Show that 
R R = C/e © U 0 . 


CHAPTER 




American mathematician Paul 
Halmos ( 1916-2006 ), who in 1942 
published the first modern linear 
algebra book. The title of 
Halmos ’s book was the same as the 
title of this chapter. 


Finite-Dimensional 
Vector Spaces 


Let’s review our standing assumptions: 

2.1 Notation F, V 

• F denotes R or C. 

• V denotes a vector space over F. 

In the last chapter we learned about vector spaces. Linear algebra focuses 
not on arbitrary vector spaces, but on finite-dimensional vector spaces, which 
we introduce in this chapter. 

LEARNING OBJECTIVES FOR THIS CHAPTER 

■ span 

■ linear independence 

■ bases 

■ dimension 
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Span and Linear Independence 

We have been writing lists of numbers surrounded by parentheses, and we will 
continue to do so for elements of F n ; for example, (2, —7, 8) € F 3 . However, 
now we need to consider lists of vectors (which may be elements of F” or of 
other vector spaces). To avoid confusion, we will usually write lists of vectors 
without surrounding parentheses. For example, (4, 1, 6), (9, 5, 7) is a list of 
length 2 of vectors in R 3 . 

2.2 Notation list of vectors 

We will usually write lists of vectors without surrounding parentheses. 


2.A 


Linear Combinations and Span 

Adding up scalar multiples of vectors in a list gives what is called a linear 
combination of the list. Here is the formal definition: 

2.3 Definition linear combination 

A linear combination of a list vi v m of vectors in V is a vector of 

the form 

tfivi + ■ ■ ■ + a m v m , 

where a\,...,a m e F. 


2.4 Example InF 3 , 

• (17, —4, 2) is a linear combination of (2, 1, —3), (1,-2. 4) because 

(17, -4, 2) = 6(2, 1,-3) + 5(1, -2. 4). 

• (17, —4, 5) is not a linear combination of (2, 1, —3), (1, —2, 4) because 
there do not exist numbers a\ , <22 e F such that 

(17, -4, 5) = fli(2, 1, -3) + a 2 ( 1,-2, 4). 

In other words, the system of equations 

17 = 2a\ + a 2 
— 4 = a\ — la 2 
5 — — 3a 1 + 4a 2 

has no solutions (as you should verify). 
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2.5 Definition span 

The set of all linear combinations of a list of vectors vi , . . . , v m in V is 
called the span of vi, . . . , v m , denoted span(vi v m ). In other words, 

span(vi,. . . ,v m ) = {flivi H h a m v m : a i a m e F}. 

The span of the empty list ( ) is defined to be {0}. 


2.6 Example The previous example shows that in F 3 , 

• (17, -4, 2) e span ((2, 1, -3), (1,-2, 4)); 

• (17, -4, 5) £ span ((2, 1,-3), (1,-2, 4)). 

Some mathematicians use the term linear span, which means the same as 
span. 

2.7 Span is the smallest containing subspace 

The span of a list of vectors in V is the smallest subspace of V containing 
all the vectors in the list. 

Proof Suppose vi , . . . , v m is a list of vectors in V. 

First we show that span(vi, . . . ,v m ) is a subspace of V. The additive 
identity is in span(vi , . . . , v m ), because 

0 = Ovi + • • • + 0v m . 

Also, span(vi, . . . , v m ) is closed under addition, because 

(c/ 1 v t H \-a m Vm)+(c\vi-\ P c m v m ) = (a\+c\)v\-\ F {o m +c m )v m . 

Furthermore, span(vi, . . . , v m ) is closed under scalar multiplication, because 


A(<ai vi + • • • + a m v m ) — Xa\v\ + • • ■ + Xa m v m . 

Thus span(vi , . . . , v m ) is a subspace of V (by 1.34). 

Each Vj is a linear combination of vi, . . . , v m (to show this, set a j — 1 
and let the other a’s in 2.3 equal 0). Thus span(vi , . . . , v m ) contains each vj . 
Conversely, because subspaces are closed under scalar multiplication and 
addition, every subspace of V containing each vj contains span(vi, . . . , v m ). 
Thus span(i’i , . . . , v m ) is the smallest subspace of V containing all the vectors 
vi,...,v m . ■ 
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2.8 Definition spans 

If span(vi , . . . , i’ m ) equals V, we say that vi ,v m spans V. 


2.9 Example Suppose n is a positive integer. Show that 

(1,0 0), (0,1,0 0),...,(0,...,0,1) 

spans F". Here the j lh vector in the list above is the n -tuple with 1 in the j lh 
slot and 0 in all other slots. 

Solution Suppose (xi x n ) sF". Then 

(.* *1, ... , Xn) = xi(l, 0, . . . , 0) + X2(0, 1, 0, . . . , 0) 4 h x n (0 — , 0, 1). 

Thus (xi, . . . , x n ) € span((l, 0, . . . , 0), (0, 1, 0 0), . . . , (0, . . . , 0, 1)), as 

desired. 


Now we can make one of the key definitions in linear algebra. 

2.10 Definition finite-dimensional vector space 

A vector space is called finite-dimensional if some list of vectors in it 
spans the space. 


Example 2.9 above shows that F” 
is a finite-dimensional vector space for 
every positive integer n . 
is no doubt already familiar to you. 

2.11 Definition polynomial, V (F) 

• A function p : F — > F is called a polynomial with coefficients in F 
if there exist ao , ,a m € F such that 

p(z) = a 0 + a i z + a 2 z 2 H b a m z m 

for all z € F. 

• V(F) is the set of all polynomials with coefficients in F. 


Recall that by definition every list 
has finite length. 


The definition of a polynomial 
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With the usual operations of addition and scalar multiplication, P(F) is a 
vector space over F, as you should verify. In other words, V(F) is a subspace 
of F f , the vector space of functions from F to F. 

If a polynomial (thought of as a function from F to F) is represented by 
two sets of coefficients, then subtracting one representation of the polynomial 
from the other produces a polynomial that is identically zero as a function 
on F and hence has all zero coefficients (if you are unfamiliar with this fact, 
just believe it for now; we will prove it later — see 4.7). Conclusion: the 
coefficients of a polynomial are uniquely determined by the polynomial. Thus 
the next definition uniquely defines the degree of a polynomial. 

2.12 Definition degree of a polynomial, deg p 

• A polynomial p e 'P(F) is said to have degree m if there exist 

scalars «t d m e F with a m 0 such that 

p(z) = a 0 + aiz H b a m z m 

for all z € F. If p has degree m, we write deg p = m. 

• The polynomial that is identically 0 is said to have degree — oo. 

In the next definition, we use the convention that — oo < m, which means 
that the polynomial 0 is in V m (F). 

2.13 Definition V m (F) 

For m a nonnegative integer, V m (F ) denotes the set of all polynomials 

with coefficients in F and degree at most m. 

To verify the next example, note that V m (F) = span( 1 , r , . ... z m ); here 
we are slightly abusing notation by letting z k denote a function. 


2.14 Example V m (F) is a finite-dimensional vector space for each non- 
negative integer m. 


2.15 Definition infinite-dimensional vector space 
A vector space is called infinite-dimensional if it is not finite-dimensional. 
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2.16 Example Show that V(F) is infinite-dimensional. 

Solution Consider any list of elements of V(F). Let m denote the highest 
degree of the polynomials in this list. Then every polynomial in the span of 
this list has degree at most m. Thus z m+l is not in the span of our list. Hence 
no list spans V(F). Thus 'P(F) is infinite-dimensional. 


Linear Independence 

Suppose vi v m € V and v e span(vi , . . . , v m ). By the definition of span, 

there exist a\, ... ,a m e F such that 


v — a\V\ + • ■ ■ + a m v m . 

Consider the question of whether the choice of scalars in the equation above 
is unique. Suppose c\, ... ,c m is another set of scalars such that 

v = CiVi H b C m v m . 

Subtracting the last two equations, we have 


0 = (<3i - ci)vi H b (a m - c m )v m . 

Thus we have written 0 as a linear combination of (vi , . . . ,v m ). If the only 
way to do this is the obvious way (using 0 for all scalars), then each aj — cj 
equals 0, which means that each aj equals Cj (and thus the choice of scalars 
was indeed unique). This situation is so important that we give it a special 
name — linear independence — which we now define. 

2.17 Definition linearly independent 

• A list i’ ] . . . . . v m of vectors in V is called linearly independent if 

the only choice of a \, . . . , a m e F that makes a\v\ -\ + a m v m 

equal 0 is a\ = ■■■ = a m =0. 

• The empty list ( ) is also declared to be linearly independent. 

The reasoning above shows that vi , . . . , v m is linearly independent if and 

only if each vector in span(vi v m ) has only one representation as a linear 

combination of vq v m . 
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2.18 Example linearly independent lists 

(a) A list v of one vector v e Lis linearly independent if and only if v ^ 0. 

(b) A list of two vectors in V is linearly independent if and only if neither 
vector is a scalar multiple of the other. 

(c) (1,0, 0, 0), (0, 1,0, 0), (0, 0, 1, 0) is linearly independent in F 4 . 

(d) The list l,z, ... ,z m is linearly independent in 'P(F) for each nonnega- 
tive integer m. 


If some vectors are removed from a linearly independent list, the remaining 
list is also linearly independent, as you should verify. 

2.19 Definition linearly dependent 

• A list of vectors in V is called linearly dependent if it is not linearly 
independent. 

• In other words, a list v\, . . . ,v m of vectors in V is linearly de- 
pendent if there exist a\,...,a m e F, not all 0, such that 
a\V\ T ■ • • T ci m v m — 0. 


2.20 Example linearly dependent lists 

• (2, 3, 1), (1, —1, 2), (7, 3, 8) is linearly dependent in F 3 because 

2(2, 3, 1) + 3(1, -1,2) + ( — 1) (7, 3, 8) = (0, 0, 0). 

• The list (2, 3, 1), (1, —1, 2), (7, 3, c) is linearly dependent in F 3 if and 
only if c — 8, as you should verify. 

• If some vector in a list of vectors in V is a linear combination of the 
other vectors, then the list is linearly dependent. (Proof: After writing 
one vector in the list as equal to a linear combination of the other 
vectors, move that vector to the other side of the equation, where it will 
be multiplied by —1.) 

• Every list of vectors in V containing the 0 vector is linearly dependent. 
(This is a special case of the previous bullet point.) 
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The lemma below will often be useful. It states that given a linearly 
dependent list of vectors, one of the vectors is in the span of the previous ones 
and furthermore we can throw out that vector without changing the span of 
the original list. 

2.21 Linear Dependence Lemma 

Suppose vj v m is a linearly dependent list in V. Then there exists 

j e {1, 2 m) such that the following hold: 

vj e span(vi, . . . , y/_i); 

if the j th term is removed from vi , . . . , v m , the span of the remain- 
ing list equals span(vi ,v m ). 

Because the list v\, ... ,v m is linearly dependent, there exist numbers 
. , a m e F, not all 0, such that 

ci \ vi + • • • + a m v m 0. 

be the largest element of {1 , ... ,m} such that aj ^ 0. Then 


proving (a). 

To prove (b), suppose u e span(yi v m ). Then there exist numbers 

c\, . . . ,c m € F such that 


(a) 

(b) 

Proof 

a i, . . 

Let j 

2.22 


U — C\V\ +■■■ + c m v m . 

In the equation above, we can replace Vj with the right side of 2.22, which 
shows that u is in the span of the list obtained by removing the j th term from 
vi v m . Thus (b) holds. ■ 

Choosing j = 1 in the Linear Dependence Lemma above means that 
vi =0, because if j = 1 then condition (a) above is interpreted to mean that 
vi e span( ); recall that span( ) = {0}. Note also that the proof of part (b) 
above needs to be modified in an obvious way if v\ =0 and j = 1 . 

In general, the proofs in the rest of the book will not call attention to 
special cases that must be considered involving empty lists, lists of length 1, 
the subspace {0}, or other trivial cases for which the result is clearly true but 
needs a slightly different proof. Be sure to check these special cases yourself. 

Now we come to a key result. It says that no linearly independent list in V 
is longer than a spanning list in V. 
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2.23 Length of linearly independent list < length of spanning list 

In a finite-dimensional vector space, the length of every linearly indepen- 
dent list of vectors is less than or equal to the length of every spanning list 
of vectors. 

Proof Suppose u\, . . . ,u m is linearly independent in V. Suppose also that 
w\, ... ,w n spans V. We need to prove that m < n. We do so through the 
multi-step process described below; note that in each step we add one of the 
m’s and remove one of the w’s. 

Step 1 

Let B be the list w\ ,w n , which spans V. Thus adjoining any vector 

in V to this list produces a linearly dependent list (because the newly 
adjoined vector can be written as a linear combination of the other 
vectors). In particular, the list 

Ml, wi, . . . ,w n 

is linearly dependent. Thus by the Linear Dependence Lemma (2.21), 
we can remove one of the w’s so that the new list B (of length n) 
consisting of m i and the remaining w’s spans V. 


Step j 

The list B (of length n) from step j — 1 spans V. Thus adjoining any 
vector to this list produces a linearly dependent list. In particular, the 
list of length (« + 1) obtained by adjoining iij to B, placing it just after 
Mi, ... , Uj—i, is linearly dependent. By the Linear Dependence Lemma 
(2.21), one of the vectors in this list is in the span of the previous ones, 
and because u\,...,Uj is linearly independent, this vector is one of 
the w’s, not one of the n’s. We can remove that w from B so that the 
new list B (of length n ) consisting of u\, ... ,Uj and the remaining w’s 
spans V. 

After step m, we have added all the m’s and the process stops. At each step as 
we add a u to B. the Linear Dependence Lemma implies that there is some w 
to remove. Thus there are at least as many w’s as m’s. ■ 

The next two examples show how the result above can be used to show, 
without any computations, that certain lists are not linearly independent and 
that certain lists do not span a given vector space. 
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2.24 Example Show that the list (1,2, 3), (4, 5, 8), (9, 6, 7), (—3, 2, 8) is 
not linearly independent in R 3 . 

Solution The list (1, 0, 0), (0, 1,0), (0, 0, 1) spans R 3 . Thus no list of length 
larger than 3 is linearly independent in R 3 . 


2.25 Example Show that the list (1,2, 3,— 5), (4, 5, 8, 3), (9, 6, 7,-1) 
does not span R 4 . 

Solution The list (1,0, 0, 0), (0, 1,0, 0), (0, 0, 1,0), (0, 0, 0, 1) is linearly in- 
dependent in R 4 . Thus no list of length less than 4 spans R 4 . 

Our intuition suggests that every subspace of a finite-dimensional vector 
space should also be finite-dimensional. We now prove that this intuition is 
correct. 

2.26 Finite-dimensional subspaces 

Every subspace of a finite-dimensional vector space is finite-dimensional. 

Proof Suppose V is finite-dimensional and U is a subspace of V. We need to 
prove that U is finite-dimensional. We do this through the following multi-step 
construction. 

Step 1 

If U = {0}, then U is finite-dimensional and we are done. If U ^ {0}, 
then choose a nonzero vector v\ € U. 


Step j 

If U — span(i'i, . . . , Vy_i), then U is finite-dimensional and we are 
done. If U ^ span(vi , . . . , v 7 _ 1 ), then choose a vector vj e U such 
that 

vj <£ span(vi,...,v 7 _i). 

After each step, as long as the process continues, we have constructed a list of 
vectors such that no vector in this list is in the span of the previous vectors. 
Thus after each step we have constructed a linearly independent list, by the 
Linear Dependence Lemma (2.21). This linearly independent list cannot be 
longer than any spanning list of V (by 2.23). Thus the process eventually 
terminates, which means that U is finite-dimensional. ■ 
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EXERCISES 2. A 

1 Suppose vi , v 2 , V 3 , V 4 spans F Prove that the list 

vi -v 2 ,v 2 -v 3 ,v 3 - v 4 ,v 4 

also spans V. 

2 Verify the assertions in Example 2. 18. 

3 Find a number t such that 

(3, 1,4), (2, -3, 5), (5, 9,0 
is not linearly independent in R 3 . 

4 Verify the assertion in the second bullet point in Example 2.20. 

5 (a) Show that if we think of C as a vector space over R, then the list 

(1 + /, 1 — i) is linearly independent. 

(b) Show that if we think of C as a vector space over C, then the list 
(1 + i, 1 — i) is linearly dependent. 

6 Suppose vi , v 2 , v 3 , v 4 is linearly independent in V. Prove that the list 

vi - v 2 , v 2 - v 3 , v 3 - v 4 , v 4 
is also linearly independent. 

7 Prove or give a counterexample: If v 3 , v 2 , . . . , v m is a linearly indepen- 
dent list of vectors in V, then 

5vi - 4 v 2 , v 2 , v 3 , . . . , v m 

is linearly independent. 

8 Prove or give a counterexample: If vi , v 2 , . . . , v m is a linearly indepen- 
dent list of vectors in F and A e F with A ^ 0, then Avi , Av 2 Xv m 

is linearly independent. 

9 Prove or give a counterexample: If v\ v m and w ]...., w m are lin- 

early independent lists of vectors in V, then v 3 + w\, . . . , v m + w m is 
linearly independent. 

10 Suppose vi , . . . , v m is linearly independent in V and w € V. Prove that 
if vi + w, . . . , v m + w is linearly dependent, then w € span(vi , . . . , v m ). 
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11 Suppose vi , . . . , v m is linearly independent in V and w € V. Show that 

Vi v m , w is linearly independent if and only if 

w £ span(vi ,v m ). 

12 Explain why there does not exist a list of six polynomials that is linearly 
independent in ^(F). 

13 Explain why no list of four polynomials spans EhlF). 

14 Prove that V is infinite-dimensional if and only if there is a sequence 
vi , V 2 , . . . of vectors in V such that tq , . . . , v m is linearly independent 
for every positive integer m. 

15 Prove that F°° is infinite-dimensional. 

16 Prove that the real vector space of all continuous real-valued functions 
on the interval [0, 1] is infinite-dimensional. 

17 Suppose po , pi p m are polynomials in V m (F) such that pj (2) = 0 

for each j . Prove that po, p\, . . . , p m is not linearly independent in 
V m ( F). ‘ 
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Bases 

In the last section, we discussed linearly independent lists and spanning lists. 
Now we bring these concepts together. 

2.27 Definition basis 

A basis of V is a list of vectors in V that is linearly independent and 
spans V. 


2.B 


2.28 

Example bases 


(a) 

The list (1 , 0, . . . , 0), (0, 1 , 0, . . . , 0), . . . , (0, . 
called the standard basis of F” . 

. . , 0, 1) is a basis of F”, 

(b) 

The list (1,2), (3, 5) is a basis of F 2 . 


(c) 

The list (1,2, —4), (7, —5, 6) is linearly independent in F 3 but is not a 
basis of F 3 because it does not span F 3 . 

(d) 

The list (1, 2), (3, 5), (4, 13) spans F 2 but is not a basis of F 2 because 
it is not linearly independent. 

(e) 

The list (1, 1,0), (0, 0, 1) is a basis of {(x, x, y) € F 3 : x, y e F}. 

(f) 

The list (1, —1, 0), (1, 0, —1) is a basis of 

{(x, y, z) e F 3 : x + y + z 

= 0}. 

(g) 

The list 1, z, . . . , z m is a basis of V m {F). 



In addition to the standard basis, F” has many other bases. For example, 
(7, 5), (—4, 9) and (1,2), (3, 5) are both bases of F 2 . 

The next result helps explain why bases are useful. Recall that “uniquely” 
means “in only one way”. 

2.29 Criterion for basis 

A list vi , . . . , v n of vectors in V is a basis of V if and only if every v € V 
can be written uniquely in the form 

2.30 v = a\v\ + ■ ■ ■ + a n v n , 


where a \, . . . , a n e F. 



40 


CHAPTER 2 Finite-Dimensional Vector Spaces 


Proof First suppose that vi , . . . , v n is a basis of V. Let v e V. Because 
vi, . . . , v„ spans V, there exist a\, . . . ,a n e F such that 2.30 holds. To 

show that the representation in 2.30 is 
unique, suppose c\, . . . ,c n are scalars 
such that we also have 


V = ClVl + 1- c n v n . 

Subtracting the last equation from 2.30, we get 

0 = (a i — ci)vi 4 F (a n - c n )v n . 

This implies that each aj — cj equals 0 (because v j v„ is linearly inde- 

pendent). Hence a\ = c\, ... ,a n — c n . We have the desired uniqueness, 
completing the proof in one direction. 

For the other direction, suppose every v € V can be written uniquely in 
the form given by 2.30. Clearly this implies that vi , . . . , v„ spans V. To show 
that vi, . . . , v n is linearly independent, suppose ci\ , . . . , a n € F are such that 

0 = a\v i + ■ ■ ■ + a n v n . 

The uniqueness of the representation 2.30 (taking v = 0) now implies that 
a i = ■ ■ ■ = a n — 0. Thus v\, ... ,v n is linearly independent and hence is a 
basis of V. ■ 

A spanning list in a vector space may not be a basis because it is not 
linearly independent. Our next result says that given any spanning list, some 
(possibly none) of the vectors in it can be discarded so that the remaining list 
is linearly independent and still spans the vector space. 

As an example in the vector space F 2 , if the procedure in the proof below 
is applied to the list (1,2), (3, 6), (4, 7), (5, 9), then the second and fourth 
vectors will be removed. This leaves (1, 2), (4, 7), which is a basis of F 2 . 

2.31 Spanning list contains a basis 

Every spanning list in a vector space can be reduced to a basis of the 
vector space. 

Proof Suppose v \ , . . . , v n spans V. We want to remove some of the vectors 
from vi, ... ,v n so that the remaining vectors form a basis of V. We do this 
through the multi-step process described below. 

Start with B equal to the list vi , . . . , v n . 


This proof is essentially a repeti- 
tion of the ideas that led us to the 
definition of linear independence. 
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Step 1 

If vi = 0, delete v\ from B. If vi ^0, leave B unchanged. 

Step j 

If Vj is in span(vi, . . . , Vy_i), delete vj from B. If vj is not in 
span(vi , . . . , Vj- 1 ), leave B unchanged. 

Stop the process after step n, getting a list B. This list B spans V because our 
original list spanned V and we have discarded only vectors that were already 
in the span of the previous vectors. The process ensures that no vector in B 
is in the span of the previous ones. Thus B is linearly independent, by the 
Linear Dependence Lemma (2.21). Hence B is a basis of V. m 

Our next result, an easy corollary of the previous result, tells us that every 
finite-dimensional vector space has a basis. 

2.32 Basis of finite-dimensional vector space 
Every finite-dimensional vector space has a basis. 

Proof By definition, a finite-dimensional vector space has a spanning list. 
The previous result tells us that each spanning list can be reduced to a basis. ■ 

Our next result is in some sense a dual of 2.31, which said that every 
spanning list can be reduced to a basis. Now we show that given any linearly 
independent list, we can adjoin some additional vectors (this includes the 
possibility of adjoining no additional vectors) so that the extended list is still 
linearly independent but also spans the space. 

2.33 Linearly independent list extends to a basis 

Every linearly independent list of vectors in a finite-dimensional vector 
space can be extended to a basis of the vector space. 

Proof Suppose u\ u m is linearly independent in a finite-dimensional 

vector space V. Let wi , . . . , w n be a basis of V. Thus the list 

U l , . . . , U m , Wj , . . . , w n 

spans V. Applying the procedure of the proof of 2.31 to reduce this list to a 

basis of V produces a basis consisting of the vectors u\ u m (none of the 

m’s get deleted in this procedure because u\, , u m is linearly independent) 
and some of the w’s. ■ 
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As an example in F 3 , suppose we start with the linearly independent 
list (2, 3, 4), (9, 6, 8). If we take wi,W 2 ,wj, in the proof above to be the 
standard basis of F 3 , then the procedure in the proof above produces the list 
(2, 3, 4), (9, 6, 8), (0, 1,0), which is a basis of F 3 . 

As an application of the result above, 
we now show that every subspace of a 
finite-dimensional vector space can be 
paired with another subspace to form a 
direct sum of the whole space. 

2.34 Every subspace of V is part of a direct sum equal to V 

Suppose V is finite-dimensional and U is a subspace of V. Then there is a 
subspace W of V such that V = U © W. 

Proof Because V is finite-dimensional, so is U (see 2.26). Thus there is 
a basis u \ u m of U (see 2.32). Of course u \, . . . , u m is a linearly in- 

dependent list of vectors in V. Hence this list can be extended to a basis 

U\ u m ,wi, . . . , w n of V (see 2.33). Let W — span(wi , . .. ,w n ). 

To prove that V — U © W, by 1 .45 we need only show that 

V = U + W and UnW = {0}. 

To prove the first equation above, suppose v e V. Then, because the list 

u i .... , u m , wi w n spans V, there exist a i , . . . , a m , b \ , . . . , b n € F such 

that 

v — a\ii\ + ■ ■ ■ + a m u m + b\w\ + • ■ ■ + b n w n . 
s — 1 - ' 

U W 

In other words, we have v = u + w, where u e U and w € W are defined as 
above. Thus v e U + W , completing the proof that V — U + W. 

To show that U IT W = {0}, suppose y e U fl W. Then there exist scalars 
a\, . . . ,a m , b\, b n e F such that 

V = a \U I + • • • + Clm u m — b\W\ + • • • + b n w n . 


Using the same basic ideas but 
considerably more advanced tools, 
the next result can be proved with- 
out the hypothesis that V is finite- 
dimensional. 


Thus 

ann 1- a m u m - b\w\ b n w n = 0. 

Because u i , . . . , u m , w \ , . . . , w n is linearly independent, this implies that 
a i = ■ ■ ■ = a m = b\ = ■■■ = b n = 0. Thus v = 0, completing the proof 
that U n W = {0}. ■ 
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EXERCISES 2.B 

1 Find all vector spaces that have exactly one basis. 

2 Verify all the assertions in Example 2.28. 

3 (a) Let U be the subspace of R 5 defined by 

U = {(xi, X2, X3, X4, X5) € R 5 : x\ = 3x2 andx3 = 7x4}. 

Find a basis of U. 

(b) Extend the basis in part (a) to a basis of R 5 . 

(c) Find a subspace W of R 5 such that R 5 = U © W. 

4 (a) Let U be the subspace of C 5 defined by 

U = {(zi, Z2, Z3, Z4, Z5) e C 5 : 6zi = Z2 and Z3 + 2Z4 + 3Z5 = 0}. 
Find a basis of U. 

(b) Extend the basis in part (a) to a basis of C 5 . 

(c) Find a subspace W of C 5 such that C 5 = U © W. 

5 Prove or disprove: there exists a basis po, p \ , P2, P3 of 7^3 (F) such that 
none of the polynomials po, p \ , P2, pi has degree 2. 

6 Suppose vi , V2, V3, V4 is a basis of V. Prove that 


vi + v 2 .v 2 + v 3 ,v 3 + v 4 ,v 4 


is also a basis of V. 

7 Prove or give a counterexample: If vi , V2, V3, V4 is a basis of V and U 
is a subspace of V such that vi , V2 6 U and V3 ^ U and V4 ^ U, then 
vi, V2 is a basis of U. 

8 Suppose U and W are subspaces of V such that V — U © W. Suppose 
also that ui, . . . ,u m is a basis of U and w\, . . . ,w n is a basis of W. 
Prove that 

U\, . . . , 


is a basis of V. 
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Dimension 


Although we have been discussing finite-dimensional vector spaces, we have 
not yet defined the dimension of such an object. How should dimension be 
defined? A reasonable definition should force the dimension of F” to equal n . 
Notice that the standard basis 

( 1,0 0 ), ( 0 , 1,0 0 ),...,( 0 ,..., 0 , 1 ) 

of F" has length n. Thus we are tempted to define the dimension as the length 
of a basis. However, a finite-dimensional vector space in general has many 
different bases, and our attempted definition makes sense only if all bases in a 
given vector space have the same length. Fortunately that turns out to be the 
case, as we now show. 

2.35 Basis length does not depend on basis 

Any two bases of a finite-dimensional vector space have the same length. 

Proof Suppose V is finite-dimensional. Let B\ and B 2 be two bases of V. 
Then B\ is linearly independent in V and B 2 spans V, so the length of B\ is 
at most the length of B 2 (by 2.23). Interchanging the roles of B\ and /L, we 
also see that the length of B 2 is at most the length of B\ . Thus the length of 
B\ equals the length of B 2 , as desired. ■ 

Now that we know that any two bases of a finite-dimensional vector space 
have the same length, we can formally define the dimension of such spaces. 

2.36 Definition dimension, dim V 

• The dimension of a finite-dimensional vector space is the length of 
any basis of the vector space. 

• The dimension of V (if V is finite-dimensional) is denoted by dim V. 


2.C 


2.37 Example dimensions 

• dim F” = n because the standard basis of F” has length n . 

• dim V m (F) = m + 1 because the basis 1 ,z,...,z m of V m (F) has 
length m + 1 . 
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Every subspace of a finite-dimensional vector space is finite -dimensional 
(by 2.26) and so has a dimension. The next result gives the expected inequality 
about the dimension of a subspace. 

2.38 Dimension of a subspace 

If V is finite-dimensional and U is a subspace of V, then dim U < dim V. 

Proof Suppose V is finite-dimensional and U is a subspace of V. Think of a 
basis of U as a linearly independent list in V, and think of a basis of V as a 
spanning list in V. Now use 2.23 to conclude that dim U < dim V. m 

To check that a list of vectors in V 
is a basis of V, we must, according to 
the definition, show that the list in ques- 
tion satisfies two properties: it must be 
linearly independent and it must span 
V. The next two results show that if the 
list in question has the right length, then 
we need only check that it satisfies one 
of the two required properties. First we 
prove that every linearly independent 
list with the right length is a basis. 

2.39 Linearly independent list of the right length is a basis 

Suppose V is finite-dimensional. Then every linearly independent list of 

vectors in V with length dim V is a basis of V. 

Proof Suppose dim V — n and iq , . . . , v n is linearly independent in V. The 
list vi , . . . , v„ can be extended to a basis of V (by 2.33). However, every basis 
of V has length n , so in this case the extension is the trivial one, meaning that 

no elements are adjoined to vi , . . . , v n . In other words, v\ v n is a basis 

of V, as desired. ■ 


The real vector space R 2 has di- 
mension 2; the complex vector 
space C has dimension 1. As 
sets, R 2 can be identified with C 
( and addition is the same on both 
spaces, as is scalar multiplication 
by real numbers). Thus when we 
talk about the dimension of a vec- 
tor space, the role played by the 
choice of F cannot be neglected. 


2.40 Example Show that the list (5, 7), (4. 3) is a basis of F 2 . 

Solution This list of two vectors in F 2 is obviously linearly independent 
(because neither vector is a scalar multiple of the other). Note that F 2 has 
dimension 2. Thus 2.39 implies that the linearly independent list (5, 7), (4, 3) 
of length 2 is a basis of F 2 (we do not need to bother checking that it spans F 2 ). 
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2.41 Example Show that 1 , (x — 5) 2 , (x — 5) 3 is a basis of the subspace 
U of T^fR) defined by 

U = {p e V 3 (R) : p'(S) = 0}. 

Solution Clearly each of the polynomials 1, (x — 5) 2 , and (x — 5) 3 is in U. 

Suppose a,b,c eR and 

a + b(x — 5) 2 + c(x — 5) 3 = 0 

for every xeR. Without explicitly expanding the left side of the equation 
above, we can see that the left side has a cx 3 term. Because the right side has 
no x 3 term, this implies that c — 0. Because c = 0, we see that the left side 
has a bx 2 term, which implies that b — 0. Because b — c — 0, we can also 
conclude that a — 0. 

Thus the equation above implies that a = b = c = 0. Hence the list 
1, (x — 5) 2 , (x — 5) 3 is linearly independent in U. 

Thus dim U > 3. Because U is a subspace of Vj (R), we know that 
dim U < dim V 3 (R) = 4 (by 2.38). However, dim U cannot equal 4, because 
otherwise when we extend a basis of U to a basis of TMR) we would get a 
list with length greater than 4. Hence dim U = 3. Thus 2.39 implies that the 
linearly independent list 1, (x — 5) 2 , (x — 5) 3 is a basis of U. 

Now we prove that a spanning list with the right length is a basis. 

2.42 Spanning list of the right length is a basis 

Suppose V is finite-dimensional. Then every spanning list of vectors in V 
with length dim V is a basis of V. 

Proof Suppose dim V = n and vi , . . . , v n spans V. The list vj , . . . , v„ can 
be reduced to a basis of V (by 2.31). However, every basis of V has length 
77 , so in this case the reduction is the trivial one, meaning that no elements 
are deleted from vi, . . . , v n . In other words, vi, . . . , v n is a basis of V, as 
desired. ■ 

The next result gives a formula for the dimension of the sum of two 
subspaces of a finite-dimensional vector space. This formula is analogous 
to a familiar counting formula: the number of elements in the union of two 
finite sets equals the number of elements in the first set, plus the number of 
elements in the second set, minus the number of elements in the intersection 
of the two sets. 
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2.43 Dimension of a sum 

If U\ and U2 are subspaces of a finite-dimensional vector space, then 

dim([/i + U2) = dim U\ + dim U2 — dim(( 7 i n U2). 

Proof Let u \, . . . , u m be a basis of U\ H U 2 \ thus dim(f/i n U2 ) = m. Be- 
cause u 1 u m is a basis of U\ fl U2, it is linearly independent in U\. 

Hence this list can be extended to a basis u\, . . . , u m , vi, . . . , Vj of U 1 
(by 2 . 33 ). Thus dimt/i = m + j. Also extend u\,...,u m to a basis 

u 1 u m ,wi , . . . , w k of U2 ; thus dim U2 = m + A:. 

We will show that 

is a basis of U\ + U2- This will complete the proof, because then we will have 

dim(t/i + U2) = m + j + k 

— (m + j ) + (m + k) — m 

- dim U\ + dim U2 — dim(t/i fl ( 72 ). 

Clearly span(u 1 , . . . , u m , v\ Vj , wi w^) contains U\ and U2 and 

hence equals U\ + ( 72 - So to show that this list is a basis of U\ + U2 we need 
only show that it is linearly independent. To prove this, suppose 

a\Ui H b a m u m + b\V\ 4 f bj Vj + ciwi H b = 0 , 

where all the a’s, b's, and c’s are scalars. We need to prove that all the a’ s, 
b’s, and c’s equal 0 . The equation above can be rewritten as 

c\w\ H b c k w k - —a\U\ a m u m - b\v x bjvj, 

which shows that CiWi + • • • + c k w k e U\. All the w’s are in U2, so this 
implies that Cinq + • • • + c k w k € U\ fl U2 ■ Because u\, . . . , u m is a basis 
of ( 7 i n U 2 , we can write 

Citvi + • • ■ + c k w k = d\ u 1 + • • • + d m u m 

for some choice of scalars d\ , . . . , d m . But u 1 , . . . , u m , w\ , . . . , w k is linearly 
independent, so the last equation implies that all the c’s (and c/’s) equal 0 . 
Thus our original equation involving the a’s, b's, and c’s becomes 

ami 4 b a m u m + b\v\ 4 b bjVj = 0 . 

Because the list u \ u m , v \ , . . . , Vj is linearly independent, this equation 

implies that all the a’s and b's are 0 . We now know that all the a’s, b's, and 
c’s equal 0 , as desired. ■ 
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EXERCISES 2.C 


1 Suppose V is finite-dimensional and U is a subspace of V such that 
dim U — dim V. Prove that U — V. 

2 Show that the subspaces of R 2 are precisely {0}, R 2 , and all lines in R 2 
through the origin. 

3 Show that the subspaces of R 3 are precisely {0}, R 3 , all lines in R 3 
through the origin, and all planes in R 3 through the origin. 

4 (a) Let U — {p e 7^4 (F) : p{ 6 ) = 0}. Find a basis of U. 

(b) Extend the basis in part (a) to a basis of 7^4 (F). 

(c) Find a subspace W of 7^4 (F) such that V 4 (F) = U © W. 

5 (a) Let U — {p e 7 ^ 4 (R) : p" (6) = 0}. Find a basis of U. 

(b) Extend the basis in part (a) to a basis of TLtlR). 

(c) Find a subspace W of 7^4 (R) such that T^IR) — U © W. 

6 (a) Let U — {p e V 4 (F) : p{ 2) = p( 5)}. Find a basis of U. 

(b) Extend the basis in part (a) to a basis of 7^4 (F). 

(c) Find a subspace W of 7^4 (F) such that 7^4 (F) = U © W. 

7 (a) Let U = {p e T^IF) : p(2) = p{ 5) = p{ 6)}. Find a basis of U. 

(b) Extend the basis in part (a) to a basis of 7^4 (F). 

(c) Find a subspace W of 7^4 (F) such that TT* (F) = U © W. 

8 (a) Let U — {p e 7^4 (R) : /_^ p — 0}. Find a basis of U. 

(b) Extend the basis in part (a) to a basis of 7 ? 4(R). 

(c) Find a subspace W of 7 ? 4(R) such that T^IR) = U © W. 

9 Suppose vi v m is linearly independent in V and w € V. Prove that 

dimspan(vi + w, . . . , v m + w) > m — 1. 

10 Suppose po , pi, . . . , p m ^ V(F) are such that each pj has degree j . 
Prove that po , p \, . . . , p m is a basis of V m {F). 

11 Suppose that U and W are subspaces of R 8 such that dim U = 3, 
dim W = 5, and U + W — R 8 . Prove that R 8 = U © W. 
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12 Suppose U and W are both five-dimensional subspaces of R 9 . Prove 
that U n W + {0}. 

13 Suppose U and W are both 4-dimensional subspaces of C 6 . Prove that 
there exist two vectors in U D W such that neither of these vectors is a 
scalar multiple of the other. 

14 Suppose U\ U m are finite-dimensional subspaces of V. Prove that 

U\ + • • • + U m is finite-dimensional and 

dim({7i + ■ ■ ■ + Um) < dim U\ H + dim U m . 

15 Suppose V is finite-dimensional, with dim V — n > 1. Prove that there 
exist 1 -dimensional subspaces U\, . . . ,U n of V such that 

V = lh © ••• © U„. 

16 Suppose U i, . . . , U m are finite-dimensional subspaces of V such that 
U\ + • • • + U m is a direct sum. Prove that U\ © • • • © U m is finite- 
dimensional and 

dim U\ © ■ ■ ■ © U m = dim U\ + • • • + dim U m . 

[The exercise above deepens the analogy between direct sums of sub- 
spaces and disjoint unions of subsets. Specifically, compare this exercise 
to the following obvious statement: if a set is written as a disjoint union 
of finite subsets, then the number of elements in the set equals the sum of 
the numbers of elements in the disjoint subsets .] 

17 You might guess, by analogy with the formula for the number of ele- 
ments in the union of three subsets of a finite set, that if U \ , C/ 2 , U 3 are 
subspaces of a finite-dimensional vector space, then 

dim([/i + U 2 + U 3 ) 

= dim U\ + dim U 2 + dim U 3 

- dim(t/i n U 2 ) - dim(C/i n if ) - dim (U 2 n if ) 

+ dim(f/i n u 2 n U3). 

Prove this or give a counterexample. 





CHAPTER 

3 






German mathematician Carl 
Friedrich Gauss ( 1777-1855 ), who 
in 1809 published a method for 
solving systems of linear equations. 
This method, now called Gaussian 
elimination, was also used in a 
Chinese book published over 1600 
years earlier. 


Linear Maps 


So far our attention has focused on vector spaces. No one gets excited about 
vector spaces. The interesting part of linear algebra is the subject to which we 
now turn — linear maps. 

In this chapter we will frequently need another vector space, which we will 
call W, in addition to V. Thus our standing assumptions are now as follows: 

3-1 Notation F, V, W 

• F denotes R or C. 

• V and W denote vector spaces over F. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ Fundamental Theorem of Linear Maps 

■ the matrix of a linear map with respect to given bases 

■ isomorphic vector spaces 

■ product spaces 

■ quotient spaces 

■ the dual space of a vector space and the dual of a linear map 
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3.A 


The Vector Space of Linear Maps 


Definition and Examples of Linear Maps 

Now we are ready for one of the key definitions in linear algebra. 


3.2 Definition linear map 

A linear map from V to W is a function T : V — >• W with the following 
properties: 

additivity 

T(u + v) = T a + T v for all u , v e V ; 

homogeneity 

T(Av) = A (Tv) for all A e F and all v e V. 

Note that for linear maps we often 
use the notation T v as well as the more 
standard functional notation T(v). 


Some mathematicians use the 
term linear transformation, which 
means the same as linear map. 


3.3 Notation C(V, W) 

The set of all linear maps from V to W is denoted C(V. W). 


Let’s look at some examples of linear maps. Make sure you verify that 
each of the functions defined below is indeed a linear map: 


3.4 Example linear maps 
zero 

In addition to its other uses, we let the symbol 0 denote the function that takes 
each element of some vector space to the additive identity of another vector 
space. To be specific, 0 e £(K W) is defined by 

Ov = 0. 

The 0 on the left side of the equation above is a function from V to W, whereas 
the 0 on the right side is the additive identity in W. As usual, the context 
should allow you to distinguish between the many uses of the symbol 0. 

identity 

The identity map, denoted I, is the function on some vector space that takes 
each element to itself. To be specific, I € C{V, V) is defined by 

Iv = v. 



SECTION 3. A The Vector Space of Linear Maps 53 


differentiation 

Define D e C{P( R).'P(R)) by 

Dp = p'. 

The assertion that this function is a linear map is another way of stating a basic 
result about differentiation: (/ + g)' = f + g' and (A/)' = A /' whenever 
/, g are differentiable and A is a constant. 


integration 

Define T e £(P(R),R) by 


Tp 


f 


p(x) dx. 


The assertion that this function is linear is another way of stating a basic result 
about integration: the integral of the sum of two functions equals the sum 
of the integrals, and the integral of a constant times a function equals the 
constant times the integral of the function. 


multiplication by x 2 

Define T e C{V( R),V(K)) by 

(' Tp)(x ) = x 2 p(x) 

for x e R. 


backward shift 

Recall that F°° denotes the vector space of all sequences of elements of F. 
Define T e ^(F 00 ^ 00 ) by 

T(x l,X 2 ,X3, ...) - (X2,X 3 , . . .). 

from R 3 to R 2 

Define T e £(R 3 ,R 2 ) by 

T{x, y, z) — (2x — y + 3z, lx + 5y — 6 z). 

from F" to Y m 

Generalizing the previous example, let m and n be positive integers, let 
Ajp e F for j = 1 m and k = 1 and define T € £(F n , F m ) by 

T (x i , . . . , Xfi) — (A p i X] + • • • + A ] ?n x n , , ^4m,t^i + ■ ■ ■ + 

Actually every linear map from F" to F'” is of this form. 


The existence part of the next result means that we can find a linear map 
that takes on whatever values we wish on the vectors in a basis. The uniqueness 
part of the next result means that a linear map is completely determined by its 
values on a basis. 
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3.5 Linear maps and basis of domain 

Suppose vi v n is a basis of V and w\,...,w n € W. Then there exists 

a unique linear map T : V — » W such that 


for each j — l, ... ,n. 

Proof First we show the existence of a linear map T with the desired property. 
Define T : V — »■ W by 

T(c\v\ + ■ • • + c n v n ) — ciwi + • • • + c n w n , 

where c\, ... ,c n are arbitrary elements of F. The list vi , . . . , v n is a basis 
of V, and thus the equation above does indeed define a function T from 
V to W (because each element of V can be uniquely written in the form 
ClVl + • • • + C n v n ). 

For each j , taking Cj — 1 and the other c’s equal to 0 in the equation 
above shows that T vj — Wj . 

If u. v e V with u = airq + ■ ■ ■ + a n v n and v = Cirq + ■ • • + c n v n , then 

T(u + v) = T((ai + ci)vi H F (a„ + c n )v n ) 

= (a i + ci)wi H F (a„ + c n )w„ 

— ifl twi + • • • + ci n w n ) + (ciwi + • • • + c n w n ) 

— Tu + Tv. 

Similarly, if A e F and v = civi + • • • + c n v n , then 

T (Av) = T{Xc\v\ H F Ac„v„) 

= Aciwi + • • • + A c n w n 
= A(ciwi + • • • + c n w n ) 

= A Tv. 

Thus T is a linear map from V to W. 

To prove uniqueness, now suppose that T e C(V, W) and that T v j = Wj 
for j = \, ... ,n. Let c\. ... ,c n e F. The homogeneity of T implies that 
T(cjVj) — CjWj for j — l, ... ,n. The additivity of T now implies that 

T(c lVl + • • • + c n v n ) = C\W\ + • • • + c n w n . 

Thus T is uniquely determined on span(vi , . . . , v n ) by the equation above. 
Because iq , . . . , v n is a basis of V, this implies that T is uniquely determined 
on V. m 
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Algebraic Operations on C(V, W) 

We begin by defining addition and scalar multiplication on C( V, W ) . 


3.6 Definition addition and scalar multiplication on C(V, W) 

Suppose S,T e £(U, W) and A e F. The sum S + T and the product 
XT are the linear maps from V to W defined by 

(S + T)(v) = Sv + Tv and (XT)(v) = X (Tv) 

for all v € V. 


You should verify that S + T and XT 
as defined above are indeed linear maps. 
In other words, if S, T e C( V, W) and 
A e F, then S + T e C(V, W) and 
XT e C(V, W). 

Because we took the trouble to de- 
fine addition and scalar multiplication 
on C(V, W ) . the next result should not 
be a surprise. 


Although linear maps are perva- 
sive throughout mathematics, they 
are not as ubiquitous as imagined 
by some confused students who 
seem to think that cos is a linear 
map from R to R when they write 
that cos 2x equals 2 cos x and that 
cos(x + y ) equals cos x + cos y. 


3.7 C(V, W) is a vector space 

With the operations of addition and scalar multiplication as defined above, 

C{V, W) is a vector space. 

The routine proof of the result above is left to the reader. Note that the 
additive identity of OV, W) is the zero linear map defined earlier in this 
section. 

Usually it makes no sense to multiply together two elements of a vector 
space, but for some pairs of linear maps a useful product exists. We will need 
a third vector space, so for the rest of this section suppose U is a vector space 
over F. 

3.8 Definition Product of Linear Maps 

If T € au, V) and S e £(U, W), then the product ST e £(U, W) is 

defined by 

(ST)(h) = S(Tu) 

for u € U. 
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In other words, ST is just the usual composition S o T of two functions, 
but when both functions are linear, most mathematicians write S T instead 
ofSoL You should verify that ST is indeed a linear map from U to W 
whenever T e C(U, V ) and 5 e C(V, W). 

Note that ST is defined only when T maps into the domain of S. 


3.9 Algebraic properties of products of linear maps 


associativity 

(TiT 2 )T 3 = 7j(T 2 7Y) 

whenever T\, T 2 , and T 3 are linear maps such that the products make 
sense (meaning that T 3 maps into the domain of 73, and 7? maps into the 
domain of Ti). 


identity 


TI = IT = T 


whenever T e C(V, W) (the first 7 is the identity map on V, and the 
second 7 is the identity map on W). 


distributive properties 


(Sx + S 2 )T = SxT + S 2 T and S(7i + T 2 ) = ST X + ST 2 


whenever T, T x ,T 2 e C{U, V ) and 5, S\,S 2 e C(V, W). 


The routine proof of the result above is left to the reader. 

Multiplication of linear maps is not commutative. In other words, it is not 
necessarily true that ST = TS, even if both sides of the equation make sense. 

3.10 Example Suppose D e T^fR)) is the differentiation map 

defined in Example 3.4 and T € £(7 7 (R), PfR)) is the multiplication by x 2 
map defined earlier in this section. Show that TD ^ DT. 

Solution We have 

((TD)p)(x) = x 2 p'(x) but ((DT)p)(x) — x 2 p\x) + 2 xp(x). 

In other words, differentiating and then multiplying by x 2 is not the same as 
multiplying by x 2 and then differentiating. 
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3.1 1 Linear maps take 0 to 0 

Suppose 7 is a linear map from V to W. Then 7 (0) = 0. 


Proof By additivity, we have 


7(0) = 7(0 + 0) = 7(0) + 7(0). 


Add the additive inverse of 7 (0) to each side of the equation above to conclude 
that 7(0) = 0. ■ 

EXERCISES 3. A 

1 Suppose b, c € R. Define 7 : R 3 -a- R 2 by 


T(x, y, z) = (2x — 4 y + 3 z + b, 6x + cxyz). 


Show that 7 is linear if and only if b — c = 0. 



Show that 7 is linear if and only if b — c = 0. 

3 Suppose 7 e 7(F", F m ). Show that there exist scalars Aj^ e F for 
j — 1 , ,m and k = l, ... ,n such that 

7(xi x n ) = {Ai \Xi + • • • + Ai, n x n , ... , A m ^\X\ + • • • + A m ^ n x n ) 

for every (x x n ) 6 F". 

[The exercise above shows that 7 has the form promised in the last item 
of Example 3.4.\ 

4 Suppose 7 e E(V, W) and vi , . . . , v m is a list of vectors in V such that 
Tvi, ... ,T v m is a linearly independent list in W. Prove that vi 

> • • • > v m 

is linearly independent. 

5 Prove the assertion in 3.7. 

6 Prove the assertions in 3.9. 
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7 Show that every linear map from a 1 -dimensional vector space to itself is 
multiplication by some scalar. More precisely, prove that if dim V = 1 
and T € C(V, V), then there exists A e F such that Tv = Av for all 
V € V. 

8 Give an example of a function (p : R 2 — > R such that 

<p(av) = acp(v ) 

for all a G R and all v G R 2 but <p is not linear. 

[The exercise above and the next exercise show that neither homogeneity 
nor additivity alone is enough to imply that a function is a linear map.] 

9 Give an example of a function (p : C — ► C such that 

cp(w + z) — <p(w) + <p(z) 

for all w, z € C but <p is not linear. (Here C is thought of as a complex 
vector space.) 

[There also exists a function cp : R — > R such that (p satisfies the additiv- 
ity condition above but <p is not linear. However, showing the existence 
of such a function involves considerably more advanced tools.] 

10 Suppose U is a subspace of V with U ^ V. Suppose S G C(U, W ) and 
S' / 0 (which means that Su ^ 0 for some u G U). Define T : V — >• W 
by 

Tv _\Sv if v € U, 

|0 if v € V and v £ U. 

Prove that T is not a linear map on V. 

11 Suppose V is finite-dimensional. Prove that every linear map on a 
subspace of V can be extended to a linear map on V. In other words, 
show that if U is a subspace of V and S e C(U, W), then there exists 
T € C(V, W) such that Tu — Su for all u e U. 

12 Suppose V is finite-dimensional with dim V > 0, and suppose W is 
infinite-dimensional. Prove that C(V, W) is infinite-dimensional. 

13 Suppose vi , . . . , v m is a linearly dependent list of vectors in V. Suppose 
also that W 7 ^ {0}. Prove that there exist w 1 , . . . , w m e W such that no 
T G C(V, W) satisfies T v^ = for each k — 1 , . . . , m. 

14 Suppose V is finite-dimensional with dim V >2. Prove that there exist 
S, T G C(V, V ) such that ST ^TS. 
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Null Spaces and Ranges 

Null Space and Injectivity 

In this section we will learn about two subspaces that are intimately connected 
with each linear map. We begin with the set of vectors that get mapped to 0. 

3.12 Definition null space, null T 

For T e C(V, W), the null space of T, denoted null T, is the subset of V 
consisting of those vectors that T maps to 0: 

null T = {v e V : Tv = 0}. 


3.B 


3.13 Example null space 

• If T is the zero map from V to W, in other words if Tv — 0 for every 
veV, then null T — V. 

• Suppose < p € £(C 3 ,F) is defined by <p(zi,z 2 , z 3 ) = z\ + 2 r 2 + 3z 3 . 
Then nulli -p — {{z\, Z 2 , Z 3 ) e C 3 : zi + 2 z 2 + 3 z 3 = 0}. A basis of 
null<p is (—2, 1,0), (—3,0, 1). 

• Suppose D € £('P(R). 'P(R)) is the differentiation map defined by 
Dp — p'. The only functions whose derivative equals the zero function 
are the constant functions. Thus the null space of D equals the set of 
constant functions. 

• Supposed e ^(TTR), T^R)) is the multiplication by x 2 map defined 
by ( T p)(x) — x 2 p(x). The only polynomial p such that x 2 p(x) — 0 
for all x € R is the 0 polynomial. Thus null T = {0}. 

• Suppose T € £(F°°, F°°) is the backward shift defined by 

T(x 1,X 2 ,XJ, . . .) = (x 2 ,X 3 , . . .). 

Clearly T(x i,X2,xj, . . .) equals 0 if and only if x 2 , x$, . . . are all 0. 
Thus in this case we have null T = {(a, 0, 0, . . . ) : a € F}. 


The next result shows that the null 
space of each linear map is a subspace 
of the domain. In particular, 0 is in the 
null space of every linear map. 


Some mathematicians use the term 
kernel instead of null space. The 
word “null” means zero. Thus the 
term “ null space” should remind 
you of the connection to 0. 
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3.14 The null space is a subspace 

Suppose T € C(V, W). Then null T is a subspace of V. 


Proof Because T is a linear map, we know that T(0) — 0 (by 3.1 1). Thus 
0 G null T. 

Suppose u, v e null T. Then 

T(u + v) = Tu + Tv = 0 + 0 = 0. 


Hence u + v e null T. Thus null T is closed under addition. 
Suppose u € null T and leF. Then 

T (Xu) = XT u = AO = 0. 


Hence Xu e null T. Thus null T is closed under scalar multiplication. 


Take another look at the null spaces 
that were computed in Example 
3.13 and note that all of them are 
subspaces. 


We have shown that null T contains 
0 and is closed under addition and scalar 
multiplication. Thus null T is a sub- 
space of V (by 1.34). ■ 


As we will soon see, for a linear map 
the next definition is closely connected to the null space. 


3.15 Definition injective 

A function T : V — »• W is called injective if Tu = T v implies u = v. 


The next result says that we can check whether a linear map is injective 
by checking whether 0 is the only vector that gets mapped to 0. As a simple 
application of this result, we see that of the linear maps whose null spaces we 
computed in 3.13, only multiplication by x 2 is injective (except that the zero 
map is injective in the special case V — {0}). 


Many mathematicians use the term 
one-to-one, which means the same 
as injective. 


The definition above could be 
rephrased to say that T is injective if 
u ^ v implies that Tu ^ T v. In other 
words, T is injective if it maps distinct 
inputs to distinct outputs. 
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3.16 Injectivity is equivalent to null space equals {0} 

Let T € £(F, W). Then T is injective if and only if null T = {0}. 

Proof First suppose T is injective. We want to prove that null T — {0}. We 
already know that {0} C null T (by 3.11). To prove the inclusion in the other 
direction, suppose v e null T. Then 

T(v) = 0 = T (0). 

Because T is injective, the equation above implies that v = 0. Thus we can 
conclude that null T — {0}, as desired. 

To prove the implication in the other direction, now suppose null T = {0}. 
We want to prove that T is injective. To do this, suppose u,v e V and 
Tn = T v. Then 

0 = Tu — Tv — T(u — v) . 

Thus u — v is in null T, which equals {0}. Hence u — v = 0, which implies 
that u = v. Hence T is injective, as desired. ■ 

Range and Surjectivity 

Now we give a name to the set of outputs of a function. 

3.17 Definition range 

For T a function from V to W, the range of T is the subset of W consisting 
of those vectors that are of the form T v for some v € V : 

range T — {Tv : v € V}. 


3.18 Example range 

• If T is the zero map from V to W, in other words if Tv — 0 for every 
v € V, then range T — {0}. 

• Suppose T e £(R 2 ,R 3 ) is defined by T(x,y ) = (2x,5 y,x + y), 
then range T — {(2x, 5y, x + v) : x, y € R}. A basis of range T is 
(2, 0,1), (0,5,1). 

• Suppose D s L('P(R), 'P(R)) is the differentiation map defined by 
Dp — p'. Because for every polynomial q e 'P(R) there exists a 
polynomial p eV( R) such that p' — q, the range of D is PfR). 
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Some mathematicians use the word 
image, which means the same as I 
range. 


The next result shows that the range 
of each linear map is a subspace of 
the vector space into which it is being 
mapped. 


3.19 The range is a subspace 

If T e C(V , W), then range T is a subspace of W. 

Proof Suppose T e C(V, W). Then 7(0) = 0 (by 3.11), which implies that 
0 € range T. 

If wi,W 2 € range T, then there exist vi , V 2 6 V such that T vi = w\ and 
Tv 2 = W 2 - Thus 


T(y i + v 2 ) = 7vi + Tv 2 = w\ + w 2 . 

Hence w\ + w 2 e range T. Thus range T is closed under addition. 

If w e range T and A e F, then there exists v e V such that Tv — w. 
Thus 

T( Av) = A Tv — Aw. 

Hence Aw e range T. Thus range T is closed under scalar multiplication. 

We have shown that range T contains 0 and is closed under addition and 
scalar multiplication. Thus range T is a subspace of W (by 1.34). ■ 

3.20 Definition surjective 

A function T : V — »• W is called surjective if its range equals W. 

To illustrate the definition above, note that of the ranges we computed in 
3.18, only the differentiation map is surjective (except that the zero map is 
surjective in the special case W — {0}. 

Whether a linear map is surjective 
depends on what we are thinking of as 
the vector space into which it maps. 

3.21 Example The differentiation map D e "PsjR)) defined 

by Dp — p' is not surjective, because the polynomial x 5 is not in the range 
of D. However, the differentiation map S e £ ('/A ( R ) , TT ( R ) ) defined by 
Sp = // is surjective, because its range equals 7^4 (R), which is now the 
vector space into which S maps. 


Many mathematicians use the term 
onto, which means the same as sur- 
jective. 
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Fundamental Theorem of Linear Maps 

The next result is so important that it gets a dramatic name. 

3.22 Fundamental Theorem of Linear Maps 

Suppose V is finite-dimensional and T e C{V. W). Then range T is 
finite-dimensional and 


dim V = dim null T + dim range T. 


Proof Let u i , . . . , u m be a basis of null T ; thus dim null T = m. The linearly 
independent list u \ , . . . , u m can be extended to a basis 


U\, ... , lt m ,V\, ... ,v n 


of V (by 2.33). Thus dim V = m + n. To complete the proof, we need only 
show that range T is finite-dimensional and dim range T = n. We will do this 
by proving that 7"vi , . . . , Tv n is a basis of range T. 

Let v € V. Because U \ , . . . , u m , vi v n spans V, we can write 

v = d\u i T***T a m u m + b\ v\ + • • • + b n v n , 


where the a’s and b ’ s arc in F. Applying T to both sides of this equation, we 
get 

Tv — b\Tv\ -I F b n Tv n , 


where the terms of the form T Uj disappeared because each u j is in null T. 
The last equation implies that 7Tq, . . . , Tv n spans range T. In particular, 
range T is finite-dimensional. 

To show Tv\, . . . ,Tv n is linearly independent, suppose c\, ... ,c n e F 
and 

C\T vi H f c n Tv n = 0. 


Then 


T(c ivi H F c n v n ) = 0. 


Lienee 


C] vi + • • • + c n v n € null T. 


Because u i , . . . , u m spans null T, we can write 


civi + • • • + c n v n — diU\ + • • • + d m u m . 


where the d ’ s are in F. This equation implies that all the c’s (and d’s) are 0 
(because u i , . . . , u m , vi , . . . , v n is linearly independent). Thus Tvi , . . . , Tv„ 
is linearly independent and hence is a basis of range T, as desired. ■ 
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Now we can show that no linear map from a finite-dimensional vector 
space to a “smaller” vector space can be injective, where “smaller” is measured 
by dimension. 

3.23 A map to a smaller dimensional space is not injective 

Suppose V and W are finite-dimensional vector spaces such that 
dim V > dim W. Then no linear map from V to W is injective. 

Proof Let T € C{V, W). Then 

dim null T — dim V — dim range T 

> dim V — dim W 

> 0 , 

where the equality above comes from the Fundamental Theorem of Linear 
Maps (3.22). The inequality above states that dim null T > 0. This means 
that null T contains vectors other than 0. Thus T is not injective (by 3.16). ■ 

The next result shows that no linear map from a finite-dimensional vector 
space to a “bigger” vector space can be suijective, where “bigger” is measured 
by dimension. 

3.24 A map to a larger dimensional space is not surjective 

Suppose V and W are finite-dimensional vector spaces such that 
dim V < dim W. Then no linear map from V to W is surjective. 

Proof Let T e C(V, W). Then 

dim range T = dim V — dim null T 

< dim V 

< dim W. 

where the equality above comes from the Fundamental Theorem of Linear 
Maps (3.22). The inequality above states that dim range T < dim W. This 
means that range T cannot equal W. Thus T is not surjective. ■ 

As we will now see, 3.23 and 3.24 have important consequences in the 
theory of linear equations. The idea here is to express questions about systems 
of linear equations in terms of linear maps. 
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3.25 Example Rephrase in terms of a linear map the question of whether 
a homogeneous system of linear equations has a nonzero solution. 

Solution 

Fix positive integers m and n, and let 
Ajk e F for j = 1 , . . . , m and 
k — 1 , ,n. Consider the homoge- 
neous system of linear equations 

n 

y ' A\k x k = o 

k= 1 


Homogeneous, in this context, 
means that the constant term on the 
right side of each equation below 
is 0. 


n 

y ' A m ,k x k = 0- 

k= 1 

Obviously xi = ■■■ = x n = 0 is a solution of the system of equations above; 
the question here is whether any other solutions exist. 

Define T : F" -* F m by 

n n 

T{X \ , . . . , X n ) = ( y ' A\k x k- ■ ■ ■ • y ' A m ,k x k) ■ 
k = 1 k = 1 

The equation T(x ,x n ) — 0 (the 0 here is the additive identity in F m , 
namely, the list of length m of all 0’s) is the same as the homogeneous system 
of linear equations above. 

Thus we want to know if null T is strictly bigger than {0}. In other words, 
we can rephrase our question about nonzero solutions as follows (by 3.16): 
What condition ensures that T is not injective? 


3.26 Homogeneous system of linear equations 

A homogeneous system of linear equations with more variables than 
equations has nonzero solutions. 

Proof Use the notation and result from the example above. Thus T is a 
linear map from F” to F m , and we have a homogeneous system of m linear 

equations with n variables X] x n . From 3.23 we see that T is not injective 

if n > m. m 

Example of the result above: a homogeneous system of four linear equa- 
tions with five variables has nonzero solutions. 
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3.27 Example Consider the question of whether an inhomogeneous sys- 
tem of linear equations has no solutions for some choice of the constant terms. 
Rephrase this question in terms of a linear map. 

Solution Fix positive integers m and n, and let Aj^ € F for j = I m 

and k — l, ... ,n. For c\, ... ,c m e F, consider the system of linear equations 

n 

X! A l,k*k = Cl 
k=l 

3.28 

n 

y ' A m,k x k — Cm- 
k= 1 

The question here is whether there is some choice of c\, . . . ,c m e F such that 
no solution exists to the system above. 

Define T : F" -* F m by 

n n 

T{x\ , . . . , Xn) = ( y ' A\jcXk , . . . , y ' A m fcXk). 
k = 1 k = 1 

The equation T(x\ x„) = (ci , . . . , c m ) is the same as the system of equa- 

tions 3.28. Thus we want to know if range T ^ F m . Flence we can rephrase 
our question about not having a solution for some choice of c\, . . . ,c m e F 
as follows: What condition ensures that T is not surjective? 


3.29 Inhomogeneous system of linear equations 

An inhomogeneous system of linear equations with more equations than 
variables has no solution for some choice of the constant terms. 


Our results about homogeneous 
systems with more variables than 
equations and inhomogeneous sys- 
tems with more equations than vari- 
ables (3.26 and 3.29) are often 
proved using Gaussian elimination. 
The abstract approach taken here 
leads to cleaner proofs. 


equations with four variables has 
stant terms. 


Proof Use the notation and result from 
the example above. Thus T is a lin- 
ear map from F” to F m , and we have a 
system of m equations with n variables 

X\ x n . From 3.24 we see that T is 

not surjective if n < in. ■ 


Example of the result above: an 
inhomogeneous system of five linear 
no solution for some choice of the con- 
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EXERCISES 3.B 


1 Give an example of a linear map T such that dim null T = 3 and 
dim range T — 2. 

2 Suppose V is a vector space and S,T e C( V, V) are such that 

range S C null T. 

Prove that (ST) 2 — 0. 

3 Suppose vi, . . . , v m is a list of vectors in V. Define T € £(F"\ V) by 

T(z\, . . . ,z m ) = z\v i + + z m v m . 

(a) What property of T corresponds to vi , . . . , v m spanning V ? 

(b) What property of T corresponds to v\, ...,v m being linearly 
independent? 

4 Show that 

{T € £(R 5 , R 4 ) : dim null T > 2} 
is not a subspace of £(R 5 , R 4 ). 

5 Give an example of a linear map T : R 4 — > R 4 such that 

range T = null T. 

6 Prove that there does not exist a linear map T : R 5 — »• R 5 such that 

range T = null T. 

7 Suppose V and W are finite-dimensional with 2 < dim V < dim W. 
Show that {T e C(V, W) : T is not injective} is not a subspace of 
C(V, W). 

8 Suppose V and W are finite-dimensional with dim V > dim W > 2. 
Show that {T e C(V, W) : T is not surjective} is not a subspace of 
C(V, W). 

9 Suppose T e C(V, W) is injective and vi , . . . , v n is linearly independent 
in V. Prove that Tvi, ... ,T v n is linearly independent in W. 
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10 Suppose spans V and T e £(V, W). Prove that the list 

T v \ .... ,T v n spans range T. 

11 Suppose Si,...,S n are injective linear maps such that Si S 2 ■ ■ ■ S n 
makes sense. Prove that S 1 S 2 ■ ■ ■ S n is injective. 

12 Suppose that V is finite-dimensional and that T e C{V, W). Prove 
that there exists a subspace U of V such that U n null T — {0} and 
range T = {Tu : u e U}. 

13 Suppose T is a linear map from F 4 to F 2 such that 

null T — {(x\ , X 2 , X 3 , X 4 ) € F 4 : x\ — 5 x 2 and X 3 — 7 x 4 }. 

Prove that T is surjective. 

14 Suppose U is a 3-dimensional subspace of R 8 and that T is a linear map 
from R 8 to R 5 such that null T — U. Prove that T is surjective. 

15 Prove that there does not exist a linear map from F 5 to F 2 whose null 
space equals 

{(xi,X 2 ,xj,X 4 , X 5 ) e F 5 : x\ — 3 x 2 and X 3 — X 4 — X 5 }. 

16 Suppose there exists a linear map on V whose null space and range are 
both finite-dimensional. Prove that V is finite-dimensional. 

17 Suppose V and W are both finite-dimensional. Prove that there exists an 
injective linear map from V to W if and only if dim V < dim W. 

18 Suppose V and W are both finite -dimensional. Prove that there exists a 
surjective linear map from V onto W if and only if dim V > dim W. 

19 Suppose V and W are finite-dimensional and that U is a subspace of V. 
Prove that there exists T e £(V, W) such that null T = U if and only if 
dim U > dim V — dim W. 

20 Suppose W is finite-dimensional and T e C(V, W). Prove that T is 
injective if and only if there exists S € C(W, V) such that ST is the 
identity map on V. 

21 Suppose V is finite-dimensional and T e £( V. W). Prove that T is 
surjective if and only if there exists S € C(W, V) such that TS is the 
identity map on W. 
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22 Suppose U and V are finite-dimensional vector spaces and S e C( V. W) 
and T e C(U,V). Prove that 

dim null ST < dim null S + dim null T. 

23 Suppose U and V are finite-dimensional vector spaces and S e C{V,W) 
and T e C(U,V). Prove that 

dim range ST < min{ dim range S , dim range T } . 

24 Suppose W is finite-dimensional and T\,T 2 G C(V. W). Prove that 
null T\ C null T 2 if and only if there exists S e C(W. W) such that 
T 2 = STi. 

25 Suppose V is finite-dimensional and T\,T 2 € C(V, W). Prove that 
range 7j C range T 2 if and only if there exists S e C(V,V) such that 
7i = T 2 S. 

26 Suppose D e £(P(R), P(R)) is such that deg Dp — (deg p) — 1 for 
every nonconstant polynomial p e 'P(R). Prove that D is surjective. 

[ The notation D is used above to remind you of the differentiation map 
that sends a polynomial p to p'. Without knowing the formula for the 
derivative of a polynomial (except that it reduces the degree by 1), you 
can use the exercise above to show that for every polynomial q e 'P(R), 
there exists a polynomial p € 'P(R) such that p' = q. \ 

27 Suppose p € 'P(R). Prove that there exists a polynomial q e 'P(R ) such 
that 5 q" + 3 q' = p. 

[This exercise can be done without linear algebra, but it’s more fun to do 
it using linear algebra.] 

28 Suppose T e C(V, W), and w 1 w m is a basis of range T. Prove that 

there exist (pi , . . . , <p m e C{V, F) such that 

Tv - cp\ (v)vr 1 H h (Pm(v)w m 

for every v e V. 

29 Suppose 1 p e C(V, F). Suppose u e V is not in null 1 p. Prove that 

V — null <p © {au : a e F}. 

30 Suppose (pi and <p 2 are linear maps from V to F that have the same null 
space. Show that there exists a constant c e F such that (p\ = ccp 2 . 

31 Give an example of two linear maps T\ and T 2 from R 5 to R 2 that have 
the same null space but are such that 7j is not a scalar multiple of T 2 . 
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Matrices 

Representing a Linear Map by a Matrix 

We know that if vi , . . . , v n is a basis of V and T : V — »• W is linear, then the 

values of Tv\ T v n determine the values of T on arbitrary vectors in V 

(see 3.5). As we will soon see, matrices are used as an efficient method of 
recording the values of the T vj ’s in terms of a basis of W. 

3.30 Definition matrix, A jk 

Let m and n denote positive integers. An m-by-n matrix A is a rectangular 
array of elements of F with m rows and n columns: 


/ ^ 1,1 ■ 

A\,n 

\ 

\ A m , 1 

Am,n 

) 


The notation A jk denotes the entry in row j , column k of A. In other 
words, the first index refers to the row number and the second index refers 
to the column number. 

Thus A 2 ,3 refers to the entry in the second row, third column of a matrix A. 
3.31 Example If A = f ^ ^ 5 ~ J, then A 2 ,3 = 7. 

Now we come to the key definition in this section. 

3.32 Definition matrix of a linear map, M(T) 

Suppose T s C{V, W) and vi, . . . , v n is a basis of V and wi, . . . , w m is 
a basis of W. The matrix of T with respect to these bases is the m-by-n 
matrix M(T) whose entries A ]k are defined by 

Tv k = A\ k w\ + • • • + A mk w m . 

If the bases are not clear from the context, then the notation 
M(T, (vi, . . . , v n ), (w i, . . . ,w m )) is used. 

The matrix M.(T ) of a linear map T e C(V, W) depends on the basis 
vi , . . . , v n of V and the basis w i, ... , w m of W, as well as on T. However, the 
bases should be clear from the context, and thus they are often not included in 
the notation. 


3.C 
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To remember how M. ( T) is constructed from T, you might write across 
the top of the matrix the basis vectors vi , . . . , v n for the domain and along the 
left the basis vectors w\ , . . . , w m for the vector space into which T maps, as 
follows: 


vi ... v k ... v„ 
wi / A hk ^ 

m(t)= ; 

w m \ A m / C J 

In the matrix above only the k th col- 
umn is shown. Thus the second index 
of each displayed entry of the matrix 
above is k. The picture above should 
remind you that T v;t can be computed 
from M(T) by multiplying each entry 
in the k lh column by the correspond- 
ing Wj from the left column, and then 
adding up the resulting vectors. 

If T is a linear map from F” to F m , 
then unless stated otherwise, assume the 
bases in question are the standard ones 
(where the k lh basis vector is 1 in the 
k th slot and 0 in all the other slots). If 
you think of elements of F m as columns 
of m numbers, then you can think of the 
/: th column of M(T) as T applied to 
the k th standard basis vector. 


If T maps an n -dimensional vector 
space to an m -dimensional vector 
space, then A i(T) is an m-by-n 
matrix. 


The k' h column of M(T) con- 
sists of the scalars needed to write 
T Vk as a linear combination of 
(w\, ... , w m ): 

m 

Tv k = A j,kWj. 
i = i 


3.33 Example Suppose T e £(F 2 , F 3 ) is defined by 
T(x, y ) = (x + 3 y, 2x + 5 y, lx + 9 y). 

Find the matrix of T with respect to the standard bases of F 2 and F 3 . 

Solution Because T(\, 0) = (1,2, 7) and T( 0, 1) = (3, 5, 9), the matrix of 
T with respect to the standard bases is the 3-by-2 matrix below: 

( X 3 

M(T) =25 
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When working with V m {¥), use the standard basis 1, x, x 2 , . . . , x m unless 
the context indicates otherwise. 


3.34 Example Suppose/) e ^(/^(R), T^tR)) is the differentiation map 
defined by Dp — p'. Find the matrix of D with respect to the standard bases 
of P 3 (R) and V 2 (R). 

Solution Because ( x 11 )' = nx n ~ l , the matrix of T with respect to the 
standard bases is the 3-by-4 matrix below: 



0 10 0 


M(D) = 0 0 2 0 

\ 0 0 0 3 


Addition and Scalar Multiplication of Matrices 

For the rest of this section, assume that V and W are finite-dimensional and 
that a basis has been chosen for each of these vector spaces. Thus for each 
linear map from V to W, we can talk about its matrix (with respect to the 
chosen bases, of course). Is the matrix of the sum of two linear maps equal to 
the sum of the matrices of the two maps? 

Right now this question does not make sense, because although we have 
defined the sum of two linear maps, we have not defined the sum of two 
matrices. Fortunately, the obvious definition of the sum of two matrices has 
the right properties. Specifically, we make the following definition. 

3.35 Definition matrix addition 

The sum of two matrices of the same size is the matrix obtained by adding 
corresponding entries in the matrices: 



( 


Ai,i ... A \ ( C i,i ... C i,„ \ 


+ 
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In the following result, the assumption is that the same bases are used for 
all three linear maps S + T, S, and T. 

3.36 The matrix of the sum of linear maps 

Suppose S,T e £(V, W). Then M(S + T) = M(S) + M(T). 

The verification of the result above is left to the reader. 

Still assuming that we have some bases in mind, is the matrix of a scalar 
times a linear map equal to the scalar times the matrix of the linear map? 
Again the question does not make sense, because we have not defined scalar 
multiplication on matrices. Fortunately, the obvious definition again has the 
right properties. 

3.37 Definition scalar multiplication of a matrix 

The product of a scalar and a matrix is the matrix obtained by multiplying 

each entry in the matrix by the scalar: 



A 


In other words, (A A) J j c = A Aj^. 

In the following result, the assumption is that the same bases arc used for 
both lineal - maps A T and T. 

3.38 The matrix of a scalar times a linear map 
Suppose A e F and T e OV, W). Then M(XT) = A M(T). 

The verification of the result above is also left to the reader. 

Because addition and scalar multiplication have now been defined for 
matrices, you should not be surprised that a vector space is about to appear. 
We need only a bit of notation so that this new vector space has a name. 

3.39 Notation 

For m and n positive integers, the set of all m-by-n matrices with entries 
in F is denoted by F m,n . 
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3.40 dim F m ,n = mn 

Suppose m and n are positive integers. With addition and scalar multipli- 
cation defined as above, F m ’ n is a vector space with dimension mn. 

Proof The verification that Y m - n is a vector space is left to the reader. Note 
that the additive identity of F m ’ n is the m-by-n matrix whose entries all 
equal 0. 

The reader should also verify that the list of m-by-n matrices that have 0 
in all entries except for a 1 in one entry is a basis of¥ m,n . There are mn such 
matrices, so the dimension of F'” : " equals mn. ■ 

Matrix Multiplication 

Suppose, as previously, that iq v n is a basis of V and w i, . . . , w m is 

a basis of W. Suppose also that we have another vector space U and that 
Mi, ... , Up is a basis of U. 

Consider linear maps T : U — »• V and S : V — »• W. The composition 
ST is a linear map from U to W. Does JA(ST ) equal M(S)M(T)1 This 
question does not yet make sense, because we have not defined the product of 
two matrices. We will choose a definition of matrix multiplication that forces 
this question to have a positive answer. Let’s see how to do this. 

Suppose M.(S) = A and M.{T) = C . For 1 < k < p, we have 

n 

(ST)u k = S (Ec r,kVr) 
r= 1 
n 

= C rJc Sv r 

r= 1 

n m 

— Cr,k ^ Aj, r Wj 

r= 1 7 = 1 

m n 

= y~! ^j, r Cr,k) w j ■ 

7=1 r=l 

Thus Ad (ST) is the m-by-p matrix whose entry in row j , column k, equals 

n 

^ A j,r C r,k- 
r= 1 
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Now we see how to define matrix multiplication so that the desired equation 
M(ST ) = M(S)M(T) holds. 

3.41 Definition matrix multiplication 

Suppose A is an m-by-n matrix and C is an n-by-p matrix. Then AC is 
defined to be the in- by- p matrix whose entry in row j , column k, is given 
by the following equation: 

n 

(AC) jJc = J2 A j,rCr,k- 

r= 1 

In other words, the entry in row j, column k, of AC is computed by 
taking row j of A and column k of C , multiplying together corresponding 
entries, and then summing. 


Note that we define the product of 
two matrices only when the number of 
columns of the first matrix equals the 
number of rows of the second matrix. 


You may have learned this defini- 
tion of matrix multiplication in an 
earlier course, although you may 
not have seen the motivation for it. 


3.42 Example Here we multiply together a 3-by-2 matrix and a 2-by-4 
matrix, obtaining a 3-by-4 matrix: 



6 5 4 
2 1 0 


3 

-1 


10 7 4 1 \ 

26 19 12 5 
42 31 20 9 / 


Matrix multiplication is not commutative. In other words, AC is not 
necessarily equal to CA even if both products are defined (see Exercise 12). 
Matrix multiplication is distributive and associative (see Exercises 13 and 14). 

In the following result, the assumption is that the same basis of V is used 
in considering T € C(U, V) and S e C{V, W), the same basis of W is used 
in considering S e C{V, W) and ST e C{U, W), and the same basis of U is 
used in considering T e jC(U, V) and ST € C(U,W). 


3.43 The matrix of the product of linear maps 

If T e C(U, V) and S e C{V, W), then M(ST) = M(S)M(T). 


The proof of the result above is the calculation that was done as motivation 
before the definition of matrix multiplication. 
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In the next piece of notation, note that as usual the first index refers to a 
row and the second index refers to a column, with a vertically centered dot 
used as a placeholder. 

3.44 Notation Aj t . , A. k 
Suppose A is an m-by-n matrix. 

• If 1 < j < m, then A y. denotes the l-by-/7 matrix consisting of 
row j of A. 

• If 1 < k < n, then A. k denotes the m-by-1 matrix consisting of 
column k of A. 


( 8 

4 

5 \ 

3.45 Example If A = 1 

column 2 of A. In other words, 

9 

1, then A 2 .- is row 2 of A and A. >2 is 

A 2 , = ( 1 9 

7 ) 

and A. a = ( 9 ) • 


The product of a 1 -by-// matrix and an //-by- 1 matrix is a 1 -by- 1 matrix. 
However, we will frequently identify a 1 -by- 1 matrix with its entry. 


3.46 Example (3 4 ) ^ ^ ^ = ( 26 ) because 3 • 6 + 4 ■ 2 = 26. 

However, we can identify ( 26 ) with 26, writing (3 4 ) ( 2 ) = 26 ' 

Our next result gives another way to think of matrix multiplication: the 
entry in row j , column k, of AC equals (row j of A) times (column k of C). 

3.47 Entry of matrix product equals row times column 

Suppose A is an m-by-n matrix and C is an n-by-p matrix. Then 

(AC) Lk = Aj, C k 

for 1 < j < m and 1 < k < p. 

The proof of the result above follows immediately from the definitions. 

3.48 Example The result above and Example 3.46 show why the entry 
in row 2, column 1, of the product in Example 3.42 equals 26. 
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The next result gives yet another way to think of matrix multiplication. It 
states that column k of AC equals A times column k of C . 

3.49 Column of matrix product equals matrix times column 
Suppose A is an m-by-n matrix and C is an n-by-p matrix. Then 

(AC\ k = AC.* 

for 1 < k < p. 

Again, the proof of the result above follows immediately from the defini- 
tions and is left to the reader. 


3.50 Example From the result above and the equation 



we see why column 2 in the matrix product in Example 3.42 is the right side 
of the equation above. 


We give one more way of thinking about the product of an m-by-n matrix 
and an //-by- 1 matrix. The following example illustrates this approach. 


3.51 Example In the example above, the product of a 3-by-2 matrix and 
a 2-by- 1 matrix is a linear combination of the columns of the 3-by-2 matrix, 
with the scalars that multiply the columns coming from the 2-by-l matrix. 
Specifically, 



The next result generalizes the example above. Again, the proof follows 
easily from the definitions and is left to the reader. 


3.52 Linear combination of columns 
Suppose A is an m-by-n matrix and c — 

Then 


Cl 


is an /7-by-l matrix. 


Cn / 


Ac — C\A-i + • • • + C n A.^ n 


In other words, Ac is a linear combination of the columns of A, with the 
scalars that multiply the columns coming from c. 
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Two more ways to think about matrix multiplication are given by Exercises 


10 and 11. 


EXERCISES 3.C 


1 Suppose V and W are finite -dimensional and T € C(V. W). Show that 
with respect to each choice of bases of V and W, the matrix of T has at 
least dim range T nonzero entries. 

2 Suppose D 6 ^(^(R), ^(R)) is the differentiation map defined by 
Dp — p'. Find a basis of T^fR) and a basis of T^IR) such that the 
matrix of D with respect to these bases is 



[Compare the exercise above to Example 3.34. 

The next exercise generalizes the exercise above.] 

3 Suppose V and W are finite-dimensional and T e C( V. W). Prove 
that there exist a basis of V and a basis of W such that with respect to 
these bases, all entries of M(T) are 0 except that the entries in row j , 
column j , equal 1 for 1 <j < dim range T. 

4 Suppose vi , . . . , v m is a basis of V and W is finite-dimensional. Suppose 
T € C(V, W). Prove that there exists a basis w \ , . . . , w„ of IP such that 
all the entries in the first column of M.(T) (with respect to the bases 

vi, ... ,v m and w\ w n ) are 0 except for possibly a 1 in the first row, 

first column. 

[In this exercise, unlike Exercise 3, you are given the basis of V instead 
of being able to choose a basis of V. ] 

5 Suppose wi , . . . , w n is a basis of W and V is finite-dimensional. Suppose 
T € C{V , W). Prove that there exists a basis vj, . . . , v m of V such 
that all the entries in the first row of M.(T) (with respect to the bases 
vi, ... ,v m and w \ , . . . , w n ) are 0 except for possibly a 1 in the first row, 
first column. 

[In this exercise, unlike Exercise 3, you are given the basis of W instead 
of being able to choose a basis of W.] 
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6 Suppose V and W are finite-dimensional and T e C( V, W ). Prove that 
dim range T = 1 if and only if there exist a basis of V and a basis of W 
such that with respect to these bases, all entries of M.{T ) equal 1. 

7 Verify 3.36. 

8 Verify 3.38. 

9 Prove 3.52. 

10 Suppose A is an in -by -n matrix and C is an //-by- p matrix. Prove that 

MO/, = Aj, C 

for 1 < j < m. In other words, show that row j of AC equals 
(row j of A) times C . 

11 Suppose a = ( a\ ■■■ a n ) is a 1-by-n matrix and C is an n-by-p 
matrix. Prove that 

aC = ci j Ci,. -(- • • • -(- CL n C n . . 

In other words, show that aC is a linear combination of the rows of C, 
with the scalars that multiply the rows coming from a. 

12 Give an example with 2-by-2 matrices to show that matrix multiplication 
is not commutative. In other words, find 2-by-2 matrices A and C such 
that AC ± CA. 

13 Prove that the distributive property holds for matrix addition and matrix 
multiplication. In other words, suppose A, B , C, D, E, and F are 
matrices whose sizes are such that A(B + C) and ( D + E)F make 
sense. Prove that AB + ^4C and DF + EF both make sense and that 
A(B + C) = AB + AC and (D + E)F = DF + EF. 

14 Prove that matrix multiplication is associative. In other words, suppose 
A, B, and C are matrices whose sizes are such that {AB)C makes sense. 
Prove that A{BC) makes sense and that ( AB)C = A(BC). 

15 Suppose A is an n-by-n matrix and 1 < j,k < n. Show that the entry in 
row j , column k, of A 3, (which is defined to mean ^4^4^4) is 

n n 

'Yh A i,P A PS A r,k- 

p= 1 r = 1 
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Invertibility and Isomorphic Vector 
Spaces 

Invertible Linear Maps 

We begin this section by defining the notions of invertible and inverse in the 
context of linear maps. 

3.53 Definition invertible, inverse 

• A linear map T e C(V. W) is called invertible if there exists a 
linear map S € C(W. V ) such that ST equals the identity map on 
V and T S equals the identity map on W. 

• A linear map S € C(W, V) satisfying ST = I and TS = I is 
called an inverse of T (note that the first / is the identity map on V 
and the second I is the identity map on If). 

3.54 Inverse is unique 

An invertible linear map has a unique inverse. 

Proof Suppose T e C{V. W) is invertible and Si and Si are inverses of T. 
Then 

Si = SiI = Si(TS 2 ) = (SiT)S 2 = I S 2 = S 2 . 

Thus Si = Si. m 

Now that we know that the inverse is unique, we can give it a notation. 

3.55 Notation T~ l 

If T is invertible, then its inverse is denoted by T~ l . In other words, if 
T e C{V, W) is invertible, then T~ l is the unique element of C{ W. V ) 
such that T~ X T = / and TT~ l = I. 

The following result characterizes the invertible linear maps. 

3.56 Invertibility is equivalent to injectivity and surjectivity 

A linear map is invertible if and only if it is injective and surjective. 


3.D 
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Proof Suppose T e C(V, W). We need to show that T is invertible if and 
only if it is injective and surjective. 

First suppose T is invertible. To show that T is injective, suppose w, v e V 
and Tu — Tv. Then 

u = T~ l (Tu) = T~ l (Tv) = v, 
so u = v. Hence T is injective. 

We are still assuming that T is invertible. Now we want to prove that T is 
surjective. To do this, let w € W. Then w — T(T~ 1 w), which shows that w is 
in the range of T. Thus range T — W. Hence T is surjective, completing this 
direction of the proof. 

Now suppose T is injective and surjective. We want to prove that T is 
invertible. For each w € W, define Sw to be the unique element of V such 
that T(Sw ) = w (the existence and uniqueness of such an element follow 
from the surjectivity and injectivity of T ). Clearly T o S equals the identity 
map on W. 

To prove that S o T equals the identity map on V, let v e V. Then 

T((So T)v) — (To S)(T v) = I (Tv) = Tv. 

This equation implies that (S o T)v = v (because T is injective). Thus S o T 
equals the identity map on V. 

To complete the proof, we need to show that S is linear. To do this, suppose 
wi,W 2 € W. Then 

T(Swi + Sw 2 ) — T(Swi) + T(Sw 2 ) — wi + w 2 . 

Thus Swi + 5 'vv’ 2 is the unique element of V that T maps to wi + w 2 . By 
the definition of S, this implies that S(wi + w 2 ) = SVi + Sw 2 . Hence S 
satisfies the additive property required for linearity. 

The proof of homogeneity is similar. Specifically, if w € W and X e F, 
then 

T(XSw) = XT(Sw) = Xw. 

Thus X Sw is the unique element of V that T maps to Xw. By the definition of 
S, this implies that S(Aw) = ASw. Hence S is linear, as desired. ■ 


3.57 Example linear maps that are not invertible 

• The multiplication by x 2 linear map from 'P(R) to 'P(R) (see 3.4) is 
not invertible because it is not surjective (1 is not in the range). 

• The backward shift linear map from F°° to F°° (see 3.4) is not invertible 
because it is not injective [(1,0, 0, 0, . . . ) is in the null space]. 
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Isomorphic Vector Spaces 

The next definition captures the idea of two vector spaces that are essentially 
the same, except for the names of the elements of the vector spaces. 

3.58 Definition isomorphism, isomorphic 

• An isomorphism is an invertible linear map. 

• Two vector spaces are called isomorphic if there is an isomorphism 
from one vector space onto the other one. 

Think of an isomorphism T : V — >• W as relabeling v e V as T v e W. This 
viewpoint explains why two isomorphic vector spaces have the same vector 
space properties. The terms “isomorphism” and “invertible linear map” mean 

the same thing. Use “isomorphism" 
when you want to emphasize that the 
two spaces are essentially the same. 


The Greek word isos means equal; 
the Greek word morph means 
shape. Thus isomorphic literally 
means equal shape. 


3.59 Dimension shows whether vector spaces are isomorphic 

Two finite-dimensional vector spaces over F arc isomorphic if and only if 
they have the same dimension. 

Proof First suppose V and W are isomorphic finite-dimensional vector 
spaces. Thus there exists an isomorphism T from V onto W. Because T is 
invertible, we have null T — {0} and range T — W. Thus dim null T — 0 
and dim range T — dim W. The formula 

dim V = dim null T + dim range T 

(the Fundamental Theorem of Linear Maps, which is 3.22) thus becomes the 
equation dim V — dim W, completing the proof in one direction. 

To prove the other direction, suppose V and W are finite-dimensional 
vector spaces with the same dimension. Let v\, ... ,v n be a basis of V and 
wi , . . . , w n be a basis of W. Let T e C{V, W) be defined by 

T(c 1V1 H F C n Vn) - C\W\ -f F c„w„. 

Then T is a well-defined linear map because vi , . . . , v n is a basis of V 

(see 3.5). Also, T is surjective because wi w n spans W. Furthermore, 

null T — {0} because w\ w n is linearly independent; thus T is injective. 

Because T is injective and surjective, it is an isomorphism (see 3.56). Hence 
V and W are isomorphic, as desired. ■ 
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The previous result implies that each 
finite-dimensional vector space V is iso- 
morphic to F" , where n = dim V. 

If vi, . . . , v n is a basis of V and 
wi , . . . , w m is a basis of W, then for 
each T e C{V, W), we have a matrix 
M(T) e F m ’ n . In other words, once 
bases have been fixed for V and W, 
A4 becomes a function from C(V. W) 
to F m,n . Notice that 3.36 and 3.38 show 
that M. is a linear map. This linear map 
is actually invertible, as we now show. 


Because every finite -dimensional 
vector space is isomorphic to some 
F", why not just study F" instead of 
more general vector spaces? To an- 
swer this question, note that an in- 
vestigation of F” would soon lead 
to other vector spaces. For exam- 
ple, we would encounter the null 
space and range of linear maps. Al- 
though each of these vector spaces 
is isomorphic to some F", thinking 
of them that way often adds com- 
plexity but no new insight. 


3.60 C(V, W) and F m ’ n are isomorphic 

Suppose vi, . . . , v n is a basis of V and wi, . . . , w m is a basis of W. 
Then A4 is an isomorphism between C( V. W) and F m,n . 

Proof We already noted that A4 is linear. We need to prove that M. is injec- 
tive and surjective. Both are easy. We begin with injectivity. If T e C(V. W) 

and A4(T ) = 0, then Tv^ = 0 for k = 1, . . . ,n. Because vi v n is a 

basis of V, this implies T — 0. Thus M. is injective (by 3.16). 

To prove that M. is surjective, suppose A e F m,n . Let T be the linear map 
from V to W such that 

m 

Tv k = 

7 = 1 

for k — 1 (see 3.5). Obviously A 4(T) equals A, and thus the range of 
A4 equals F m ,n , as desired. ■ 

Now we can determine the dimension of the vector space of linear maps 
from one finite-dimensional vector space to another. 

3.61 dim C(V, W ) = (dim F)(dim W) 

Suppose V and W are finite-dimensional. Then C(V. W ) is finite- 
dimensional and 


dim£(K, W) = (dim L)(dim W). 


Proof This follows from 3.60, 3.59, and 3.40. 
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Linear Maps Thought of as Matrix Multiplication 

Previously we defined the matrix of a linear map. Now we define the matrix 
of a vector. 

3.62 Definition matrix of a vector, M(v) 

Suppose v e V and vi, . . . ,v n is a basis of V. The matrix of v with 
respect to this basis is the n-by-1 matrix 



\ t-n 

where ci, ... ,c n are the scalars such that 


V = Cl Vi + b c n v n . 


The matrix M. (v) of a vector v e V depends on the basis v\, ... ,v n of V, 
as well as on v. However, the basis should be clear from the context and thus 
it is not included in the notation. 


3.63 Example matrix of a vector 

• The matrix oil — lx + 5x 3 with respect to the standard basis of V 3 (R) 


is 


( 2 \ 


-7 

0 


V 5 


• The matrix of a vector x e F” with respect to the standard basis is 
obtained by writing the coordinates of x as the entries in an n-by-1 
matrix. In other words, if x — (xi, . . . , x„) e F”, then 


M(x) 


V Xn 7 


Occasionally we want to think of elements of V as relabeled to be n-by-1 
matrices. Once a basis vi , . . . , v n is chosen, the function M that takes v e V 
to M.(v) is an isomorphism of V onto F”' 1 that implements this relabeling. 
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Recall that if A is an m-by-n matrix, then A. k denotes the k th column of 
A, thought of as an m-by-1 matrix. In the next result, A4(v k ) is computed 
with respect to the basis w\, ... , w m of W. 

3.64 M{T). k = M{y k ). 

Suppose T e C(V, W) and vi , . . . , v n is a basis of V and w\ , . . . , w m is 
a basis of W. Let 1 < k < n. Then the k th column of M.(T), which is 
denoted by M(T). k , equals M(v k ). 

Proof The desired result follows immediately from the definitions of AA(T) 
andAd(v^). ■ 

The next result shows how the notions of the matrix of a linear map, the 
matrix of a vector, and matrix multiplication fit together. 

3.65 Linear maps act like matrix multiplication 

Suppose T e C(V. W) and v € V. Suppose v\ v n is a basis of V and 

w i , . . . , w m is a basis of W. Then 

M(Tv) = M(T)M(v). 

Proof Suppose v = Civi + • • • + c„v n , where ci, . . . , c„ € F. Thus 

3.66 Tv — c\Tv\ H b c n T v n ■ 

Hence 


M(Tv) = c\M(T vi) + • ■ ■ + c„M(Tv n ) 

= Cl M(T). A + - + c n M(T)., n 
= M(T)M(v), 

where the first equality follows from 3.66 and the linearity of A4 . the second 
equality comes from 3.64, and the last equality comes from 3.52. ■ 

Each m-by-n matrix A induces a linear map from F”' 1 to F m ’ 1 , namely the 
matrix multiplication function that takes x 6 F"’ 1 to Ax € F rn ’ 1 . The result 
above can be used to think of every linear map (from one finite-dimensional 
vector space to another finite-dimensional vector space) as a matrix multi- 
plication map after suitable relabeling via the isomorphisms given by M.. 
Specifically, if T e C{V , W) and we identify v e V with M.(v) 6 F”’ 1 , then 
the result above says that we can identify T v with M.(T)M.{v). 
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Because the result above allows us to think (via isomorphisms) of each 
linear map as multiplication on F”’ 1 by some matrix A, keep in mind that the 
specific matrix A depends not only on the linear map but also on the choice 
of bases. One of the themes of many of the most important results in later 
chapters will be the choice of a basis that makes the matrix A as simple as 
possible. 

In this book, we concentrate on linear maps rather than on matrices. How- 
ever, sometimes thinking of linear maps as matrices (or thinking of matrices 
as linear maps) gives important insights that we will find useful. 

Operators 

Linear maps from a vector space to itself are so important that they get a 
special name and special notation. 

3.67 Definition operator, C(V) 

• A linear map from a vector space to itself is called an operator. 

• The notation C(V ) denotes the set of all operators on V. In other 
words, £(V) = C{V, V). 

A linear map is invertible if it is 
injective and surjective. For an op- 
erator, you might wonder whether in- 
jectivity alone, or surjectivity alone, 
is enough to imply invertibility. On 
infinite-dimensional vector spaces, neither condition alone implies invert- 
ibility, as illustrated by the next example, which uses two familiar operators 
from Example 3.4. 


3.68 Example neither injectivity nor surjectivity implies invertibility 

• The multiplication by x 2 operator on 'P(R) is injective but not surjective. 

• The backward shift operator on F°° is surjective but not injective. 

In view of the example above, the next result is remarkable — it states 
that for operators on a finite-dimensional vector space, either injectivity or 
surjectivity alone implies the other condition. Often it is easier to check that 
an operator on a finite-dimensional vector space is injective, and then we get 
surjectivity for free. 


The deepest and most important 
parts of linear algebra, as well as 
most of the rest of this book, deal 
with operators. 
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3.69 Injectivity is equivalent to surjectivity in finite dimensions 

Suppose V is finite-dimensional and T € C( V ). Then the following are 
equivalent: 


(a) T is invertible; 

(b) T is injective; 

(c) T is surjective. 

Proof Clearly (a) implies (b). 

Now suppose (b) holds, so that T is injective. Thus null T = {0} (by 3.16). 
From the Fundamental Theorem of Linear Maps (3.22) we have 

dim range T = dim V — dim null T 
— dim V. 


Thus range T equals V. Thus T is surjective. Flence (b) implies (c). 

Now suppose (c) holds, so that T is surjective. Thus range T = V. From 
the Fundamental Theorem of Linear Maps (3.22) we have 

dim null T = dim V — dim range T 
= 0. 

Thus null T equals {0}. Thus T is injective (by 3.16), and so T is invertible 
(we already knew that T was surjective). Flence (c) implies (a), completing 
the proof. ■ 

The next example illustrates the power of the previous result. Although 
it is possible to prove the result in the example below without using linear 
algebra, the proof using linear algebra is cleaner and easier. 


3.70 Example Show that for each polynomial q 6 'P(R), there exists a 
polynomial p e V(R) with ((x 2 + 5x + l)p)" = q. 

Solution Example 3.68 shows that the magic of 3.69 does not apply to the 
infinite-dimensional vector space T’(R). Flowever, each nonzero polynomial 
q has some degree m. By restricting attention to V m (R), we can work with a 
finite-dimensional vector space. 

Suppose q e V m (R). Define T : V m (R) — V m (R) by 

Tp = ((x 2 + 5x + l)p)" . 
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Multiplying a nonzero polynomial by (x 2 + 5x + 7) increases the degree 
by 2, and then differentiating twice reduces the degree by 2. Thus T is indeed 
an operator on V m (R)- 

Every polynomial whose second derivative equals 0 is of the form ax + b, 
where a, b e R. Thus null T — {0}. Hence T is injective. 

Now 3.69 implies that T is surjective. Thus there exists a polynomial 
p e V m (R ) such that ((x 2 + 5x + 7) p) = q, as desired. 

Exercise 30 in Section 6. A gives a similar but more spectacular application 
of 3.69. The result in that exercise is quite difficult to prove without using 
linear algebra. 

EXERCISES 3.D 


1 Suppose T e £{U, V) and S e £( V. W) are both invertible linear maps. 
Prove that ST e £(17, W) is invertible and that (5T) _1 = T~ l S' -1 . 

2 Suppose V is finite-dimensional and dim V > 1. Prove that the set of 
noninvertible operators on V is not a subspace of £( V). 

3 Suppose V is finite-dimensional, U is a subspace of V, and S e £(17, V). 
Prove there exists an invertible operator T € C(V) such that Tu — Su 
for every u € U if and only if S is injective. 

4 Suppose W is finite-dimensional and 7j , £2 G £(E, W). Prove that 
null 7j = null £2 if and only if there exists an invertible operator 
.S s C{W) such that T x = ST 2 . 

5 Suppose V is finite-dimensional and T\,T 2 G C(V. W). Prove that 
range T\ — range T 2 if and only if there exists an invertible operator 
5 e OV) such that T\ = T 2 S. 

6 Suppose V and W are finite-dimensional and T\,T 2 e £(V. W). Prove 
that there exist invertible operators R e £{ V) and S e £ ( ll 7 ) such that 
7j = ST 2 R if and only if dim null T\ — dim null T 2 . 

7 Suppose V and W are finite-dimensional. Let v e V. Let 

E = {T € aV,W) : Tv = 0}. 

(a) Show that £ is a subspace of £(F, W). 

(b) Suppose v ^ 0. What is dim El 
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Suppose V is finite-dimensional and T : V — »■ W is a surjective linear 
map of V onto W. Prove that there is a subspace U of V such that 
T\ v is an isomorphism of U onto W. (Here T \ u means the function T 
restricted to U. In other words, T\u is the function whose domain is U, 
with T\u defined by T\u(u) — Tu for every u € U.) 

Suppose V is finite-dimensional and S,T € C{V). Prove that S T is 
invertible if and only if both S and T are invertible. 

Suppose V is finite-dimensional and S.T € C(V). Prove that ST = I 
if and only if TS — I. 

Suppose V is finite-dimensional and S,T.U € C(V) and STU = I. 
Show that T is invertible and that T~ x = LIS . 

Show that the result in the previous exercise can fail without the hypoth- 
esis that V is finite-dimensional. 

Suppose V is a finite-dimensional vector space and R, S,T € C{V) are 
such that RST is surjective. Prove that S is injective. 

Suppose vi, . . . , v n is a basis of V. Prove that the map T : V — > F”’ 1 
defined by 

Tv = M(v) 

is an isomorphism of V onto F”’ 1 ; here M.(y ) is the matrix of v e V 
with respect to the basis v\, ... ,v n . 

Prove that every linear map from F”’ 1 to F™’ 1 is given by a matrix 
multiplication. In other words, prove that if T e £(F" ,1 ,F m ’ 1 ), then 
there exists an m-by-n matrix A such that T x — Ax for every x € F"’ 1 . 

Suppose V is finite-dimensional and T e C{V). Prove that T is a scalar 
multiple of the identity if and only if ST = TS for every S e C( V). 

Suppose V is finite-dimensional and £ is a subspace of C(V) such that 
ST e £ and TS e £ for all S e C{V) and all T e £. Prove that 
£ = {0} or £ — C{V). 

Show that V and £(F, V) are isomorphic vector spaces. 

Suppose T € ^(^(R)) is such that T is injective and deg Tp < deg p 
for every nonzero polynomial p e 'P(R). 

(a) Prove that T is surjective. 

(b) Prove that deg Tp — deg p for every nonzero p e 'P(R). 
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20 Suppose n is a positive integer and Aij e F for i, j — 1 Prove 
that the following are equivalent (note that in both parts below, the 
number of equations equals the number of variables): 

(a) The trivial solution x\ = ■ ■ ■ = x n = 0 is the only solution to the 
homogeneous system of equations 


n 


'y 1 o 


n 



k=l 


(b) For every c\, ... ,c n e F, there exists a solution to the system of 
equations 


n 



k= 1 


n 



k= 1 


SECTION 3.E Products and Quotients of Vector Spaces 91 


Products and Quotients of Vector Spaces 

Products of Vector Spaces 

As usual when dealing with more than one vector space, all the vector spaces 
in use should be over the same field. 

3.71 Definition product of vector spaces 
Suppose Vi, ... ,V m are vector spaces over F. 

• The product V\ x ■ ■ ■ x V m is defined by 

Vi X ■ ■ • X Vm — { (v 1 , . . . , Vm ) • V\ £ V\, ... , V m £ V m } . 

• Addition on V\ x ■ • • x V m is defined by 

(ll\ , . . . , Um) + (Vl, ...,V m ) = (u i + V\, ... ,U m + V m )- 

• Scalar multiplication on V\ x ■ ■ ■ x V m is defined by 

A(vi,...,v m ) = (Avi Av m ). 


3.E 


3.72 Example Elements of P? ( R ) x R 3 are lists of length 2, with the 
first item in the list an element of Pi(R) and the second item in the list an 
element of R 3 . 

For example, (5 — 6x + 4.x 2 , (3, 8, 7)) e P 2 (R) x R 3 . 

The next result should be interpreted to mean that the product of vector 
spaces is a vector space with the operations of addition and scalar multiplica- 
tion as defined above. 

3.73 Product of vector spaces is a vector space 

Suppose V\, . . . ,V m are vector spaces over F. Then V\ x ■ ■ ■ x V m is a 
vector space over F. 

The proof of the result above is left to the reader. Note that the additive 
identity of V\ x • ■ ■ x V m is (0, . . . , 0), where the 0 in the j th slot is the 

additive identity of V j. The additive inverse of (vi v m ) e V\ x ■ ■ ■ x V m 

is (-vi, . . . ,-v m ). 
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3.74 Example Is R 2 x R 3 equal to R 5 ? Is R 2 x R 3 isomorphic to R 5 ? 

Solution Elements of R 2 x R 3 are lists ((xi, x 2 ), (X3, X4, X5)), where 
Xl,X 2 ,X 3 ,X 4 , X 5 € R. 

Elements of R 5 are lists (xi, x 2 , X3, X4, X5), where x\, x 2 , X3, x 4 , X5 € R. 
Although these look almost the same, they are not the same kind of object. 
Elements of R 2 x R 3 are lists of length 2 (with the first item itself a list of 
length 2 and the second item a list of length 3), and elements of R 5 are lists 
of length 5. Thus R 2 x R 3 does not equal R 5 . 

The linear map that takes a vector ((xi, x 2 ), (x 3 ,x 4 , X5)) € R 2 x R 3 to 
(xi, x 2 , X3, x 4 , X5) € R 5 is clearly an isomorphism of R 2 x R 3 onto R 5 . 
Thus these two vector spaces are isomorphic. 

In this case, the isomorphism is so natural that we should think of it as a 
relabeling. Some people would even informally say that R 2 x R 3 equals R 5 , 
which is not technically correct but which captures the spirit of identification 
via relabeling. 

The next example illustrates the idea of the proof of 3.76. 

3.75 Example Find a basis of P 2 (R) x R 2 

Solution Consider this list of length 5 of elements of P 2 (R) x R 2 : 

(1, (0, 0)), (x, (0, 0)), (x 2 , (0, 0)), (0, (1,0)), (0, (0, 1)). 

The list above is linearly independent and it spans T^fR) x R 2 . Thus it is a 
basis of P 2 (R) x R 2 . 


3.76 Dimension of a product is the sum of dimensions 

Suppose V\ , ,V m are finite-dimensional vector spaces. Then 
Vi x ■ ■ ■ x V m is finite-dimensional and 


dim( F] x---x F ffl ) = dim Vi + • • • + dim V m . 


Proof Choose a basis of each Vj . For each basis vector of each V j , consider 
the element of V\ x ■ ■ ■ x V m that equals the basis vector in the j th slot and 
0 in the other slots. The list of all such vectors is linearly independent and 
spans V\ x ■ ■ ■ x V m . Thus it is a basis of V\ x ■ ■ ■ x V m . The length of this 
basis is dim V\ + • • ■ + dim V m , as desired. ■ 
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Products and Direct Sums 

In the next result, the map T is surjective by the definition of U\ + • • • + U m . 
Thus the last word in the result below could be changed from “injective” to 
“invertible”. 

3.77 Products and direct sums 

Suppose that U\ U m are subspaces of V. Define a linear map 

T : U\ x ■ ■ ■ x U m U\ + ■ ■ ■ + U m by 

T(ui,...,u m ) = Ml H 1 -Um- 

Then U\ + ■ • • + U m is a direct sum if and only if F is injective. 

Proof The linear map T is injective if and only if the only way to write 0 as a 
sum to + ■ ■ ■ + u m , where each Uj is in Uj, is by taking each Uj equal to 0. 
Thus 1.44 shows that F is injective if and only if U\ + • • ■ + U m is a direct 
sum, as desired. ■ 

3.78 A sum is a direct sum if and only if dimensions add up 

Suppose V is finite-dimensional and U\ U m are subspaces of V. Then 

U i + • • • + U m is a direct sum if and only if 


dim(£/i + • • • + U m ) — dim U\ + • ■ ■ + dim U m . 


Proof The map T in 3.77 is surjective. Thus by the Fundamental Theorem 
of Linear Maps (3.22), T is injective if and only if 


dim((7i + ■ • • + U m ) = dim(t7i x ■ ■ ■ x U m ). 


Combining 3.77 and 3.76 now shows that U\ + • • • + U m is a direct sum if 
and only if 


dim((7i + ■ ■ ■ + U m ) — dim Ci + ■ • • + dim U m , 


as desired. ■ 

In the special case m = 2, an alternative proof that U i + U 2 is a direct 
sum if and only if dim(L) + U 2 ) = dint U\ + dim U 2 can be obtained by 
combining 1.45 and 2.43. 
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Quotients of Vector Spaces 

We begin our approach to quotient spaces by defining the sum of a vector and 
a subspace. 

3.79 Definition v + U 

Suppose v € V and U is a subspace of V. Then v + U is the subset of V 
defined by 

v + U = {v + u : u € U}. 


3.80 Example Suppose 

U = {(x,2x) e R 2 : x e R), 

Then U is the line in R 2 through the origin 
with slope 2. Thus 

(17, 20) + U 

is the line in R 2 that contains the point 
(17, 20) and has slope 2. 


(10,20) (17,20) 



3.81 Definition affine subset, parallel 

• An affine subset of V is a subset of V of the form v + U for some 
v € V and some subspace U of V. 

• For v € V and U a subspace of V, the affine subset v + U is said to 
be parallel to U. 


3.82 Example parallel affine subsets 

• In Example 3.80 above, all the lines in R 2 with slope 2 are parallel to U. 

• If U = {(x, y, 0) e R 3 : x,y e R}, then the affine subsets of R 3 
parallel to U are the planes in R 3 that are parallel to the xy -plane U in 
the usual sense. 

Important: With the definition of parallel given in 3.81, no line in R 3 
is considered to be an affine subset that is parallel to the plane U. 
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3.83 Definition quotient space, V/U 

Suppose U is a subspace of V. Then the quotient space V/U is the set of 
all affine subsets of V parallel to U. In other words, 

V/U = {v + U : v e V}. 


3.84 Example quotient spaces 

• If U — {(x, 2x) e R 2 : x 6 R}, then R 2 / U is the set of all lines in 
R 2 that have slope 2. 

• If U is a line in R 3 containing the origin, then R 3 / U is the set of all 
lines in R 3 parallel to U. 

• If U is a plane in R 3 containing the origin, then R 3 / U is the set of all 
planes in R 3 parallel to U. 

Our next goal is to make V/U into a vector space. To do this, we will 
need the following result. 

3.85 Two affine subsets parallel to U are equal or disjoint 

Suppose U is a subspace of V and v,w e V. Then the following are 
equivalent: 


(a) 

V — W € U\ 




(b) 

v + U = w + U\ 




(c) 

(v + U) n (w + U) ^ 0 . 




Proof 

First suppose (a) holds, so v - 

- vv e U. I f u 

e 

U, then 


V + u = w + ((v 

— w) + U ) € 

w 

+ u. 

Thus 

v + U C w + U. Similarly, w 

+ u C V + 

U. 

Thus v + U 


completing the proof that (a) implies (b). 

Obviously (b) implies (c). 

Now suppose (c) holds, so (v + U) IT (w + U) ^ 0. Thus there exist 
U\,U 2 € U such that 

V + U 1 — W + U2- 

Thus v — w = U -2 — U\. Hence v — w e U, showing that (c) implies (a) and 
completing the proof. ■ 
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Now we can define addition and scalar multiplication on V/U. 

3.86 Definition addition and scalar multiplication on V / U 

Suppose U is a subspace of V. Then addition and scalar multiplication 
are defined on V / U by 

(v + U) + (w + U) = (v + w) + U 
A(v + U) = (Av) + U 

for v, w € V and A e F. 

As part of the proof of the next result, we will show that the definitions 
above make sense. 

3.87 Quotient space is a vector space 

Suppose U is a subspace of V. Then V/U, with the operations of addition 
and scalar multiplication as defined above, is a vector space. 

Proof The potential problem with the definitions above of addition and scalar 
multiplication on V / U is that the representation of an affine subset parallel to 
U is not unique. Specifically, suppose v, w € V. Suppose also that v, vv e V 
are such that v + U — v + U and w + U — w + U. To show that the 
definition of addition on V/U given above makes sense, we must show that 
(v T tv) T U = (v + vv) + U. 

By 3.85, we have 

v — v e U and w — w € U. 

Because U is a subspace of V and thus is closed under addition, this implies 
that (v — v) + (w — vv) € U. Thus (v + w) — (v + vv) e U. Using 3.85 again, 
we see that 

(v + w) + U — (v + vv) + [/, 

as desired. Thus the definition of addition on V/U makes sense. 

Similarly, suppose A e F. Because U is a subspace of V and thus is 
closed under scalar multiplication, we have A(v — v) e U. Thus Av — Av e U. 
ffence 3.85 implies that (Av) + U = (Av) + U. Thus the definition of scalar 
multiplication on V/U makes sense. 

Now that addition and scalar multiplication have been defined on V/U, the 
verification that these operations make V/U into a vector space is straightfor- 
ward and is left to the reader. Note that the additive identity of V/U is 0 + U 
(which equals U) and that the additive inverse of v + U is (— v) + U. ■ 
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The next concept will give us an easy way to compute the dimension 
of V/U. 


3.88 Definition quotient map, n 

Suppose U is a subspace of V. The quotient map n is the linear map 
n : V — »■ V / U defined by 

7T(v) = V + U 

for v € V. 

The reader should verify that n is indeed a linear map. Although n 
depends on U as well as V, these spaces are left out of the notation because 
they should be clear from the context. 

3.89 Dimension of a quotient space 

Suppose V is finite-dimensional and U is a subspace of V. Then 
dim V/U — dim V — dim U. 

Proof Let n be the quotient map from V to V/U. From 3.85, we see that 
null jt = U. Clearly range n — V/U. The Fundamental Theorem of Linear 
Maps (3.22) thus tells us that 

dim V = dim U + dim V/U, 

which gives the desired result. ■ 

Each linear map T on V induces a linear map T on V / (null T), which we 
now define. 

3.90 Definition f 

Suppose T € C(V, W). Define f : F/(null T) -> W by 

f(v + null T) = Tv. 


To show that the definition of T makes sense, suppose u , v € V are such 
that u + null T = v + null T. By 3.85, we have u — v e null T. Thus 
T(u — v) = 0. Flence Tu — Tv. Thus the definition of T indeed makes 


sense. 
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3.91 Null space and range of T 
Suppose T e C{V, W). Then 

(a) T is a linear map from V / (null T) to W ; 

(b) T is injective; 

(c) range T = range T ; 

(d) L/(null T) is isomorphic to range T. 

Proof 

(a) The routine verification that T is linear is left to the reader. 

(b) Suppose v € V and T (v + null T) = 0. Then Tv — 0. Thus v e null T. 
Hence 3.85 implies that v + null T — 0 + null T. This implies that 
null T — 0, and hence T is injective, as desired. 

(c) The definition of T shows that range T — range T. 

(d) Parts (b) and (c) imply that if we think of T as mapping into range T, 

then T is an isomorphism from T 1 * * * * * 7 '/ (null T) onto range T. a 


EXERCISES 3.E 


1 Suppose T is a function from V to W. The graph of T is the subset of 
V x W defined by 

graph of T = {(v. Tv) € V x W : v € V). 

Prove that T is a linear map if and only if the graph of T is a subspace 

of V x W. 

[Formally, a function T from V to W is a subset T of V x W such that 

for each v € V, there exists exactly one element (v, w ) € T. In other 
words, formally a function is what is called above its graph. We do 
not usually think of functions in this formal manner. However, if we do 
become formal, then the exercise above could be rephrased as follows: 

Prove that a function T from V to W is a linear map if and only if T is 
a subspace of V x W. \ 
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Suppose V\, ... ,V m are vector spaces such that V\ x ■ ■ ■ x V m is finite- 
dimensional. Prove that V j is finite-dimensional for each j — 1 , ... ,m. 

Give an example of a vector space V and subspaces U \ , U2 of V such 
that U\ x U2 is isomorphic to U\ + U2 but U\ + U2 is not a direct sum. 

Hint: The vector space V must be infinite-dimensional. 

Suppose Vi , . . . , V m are vector spaces. Prove that C(\'\ x ■ ■ ■ x V m . W) 
and C{V \ , W) x ■ ■ ■ x £(F m . W) are isomorphic vector spaces. 

Suppose Wi, , W m are vector spaces. Prove that C( V. W\ x ■ ■ ■ x ^n) 
and £(y, Hi ) x ■ ■ ■ x C(V. W m ) are isomorphic vector spaces. 

For n a positive integer, define V n by 

V n — V x ■■■ x V . 

' v ' 

n times 

Prove that V n and £( F", V) are isomorphic vector spaces. 

Suppose v, x are vectors in V and U, W are subspaces of V such that 
v + U = x + W. Prove that U — W. 

Prove that a nonempty subset A of V is an affine subset of V if and only 
if Av + (1 — A )w € A for all v, w e A and all A e F. 

Suppose A 1 and A 2 are affine subsets of V. Prove that the intersection 
A 1 fl A 2 is either an affine subset of V or the empty set. 

Prove that the intersection of every collection of affine subsets of V is 
either an affine subset of V or the empty set. 

Suppose vi , . . . , v m € V. Let 

A — {A1V1 + ■ • • + A m v m ■ Ai, . . . , X m € F and Ai + ■ ■ • + X m — 1 }. 

(a) Prove that A is an affine subset of V. 

(b) Prove that every affine subset of V that contains it , . . . , v m also 
contains A. 

(c) Prove that A — v + U for some v € V and some subspace U of 
V with dim U < m — 1 . 

Suppose U is a subspace of V such that V/U is finite-dimensional. 
Prove that V is isomorphic to U x (V/U). 
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13 Suppose U is a subspace of V and vi + U, . . . , v m + U is a basis of 

V/U and u \ u n is a basis of U. Prove that vi v m , u i u n 

is a basis of V. 

14 Suppose U = {(xi, X 2 , . . . ) € F°° : xj ^ 0 for only finitely many j }. 

(a) Show that U is a subspace of F°° . 

(b) Prove that F°°/ U is infinite-dimensional. 

15 Suppose cp € C(V . F) and (p ^ 0. Prove that dim F/(null <p) — 1. 

16 Suppose U is a subspace of V such that dim V/U = 1. Prove that there 
exists < p € C(V, F) such that null <p = U. 

17 Suppose U is a subspace of V such that V/U is finite-dimensional. 
Prove that there exists a subspace W of V such that dim W — dim V/U 
and V = U © W. 

18 Suppose T € C(V, W) and U is a subspace of V. Let n denote the 
quotient map from V onto V/U. Prove that there exists S e C(V/U. W) 
such that T = S o n if and only if U C null T. 

19 Find a correct statement analogous to 3.78 that is applicable to finite 
sets, with unions analogous to sums of subspaces and disjoint unions 
analogous to direct sums. 

20 Suppose U is a subspace of V. Define F : C(V/U, W) C{V , W ) by 

T(S) = Son. 

(a) Show that F is a linear map. 

(b) Show that F is injective. 

(c) Show that range F = {T € £(F, W) : Tu = 0 for every u e U}. 
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Duality 

The Dual Space and the Dual Map 

Linear maps into the scalar field F play a special role in linear algebra, and 
thus they get a special name: 

3.92 Definition linear functional 

A linear functional on V is a linear map from V to F. In other words, a 
linear functional is an element of £(F, F). 

3.93 Example linear functionals 

• Define cp : R 3 — *• R by cp(x, y, z) — Ax — 5y + 2 z. Then tp is a linear 
functional on R 3 . 

• Fix (ci , . . . , c n ) sF". Define <p : F” — >• F by 

<p(x i,...,x n ) = c ixi H F c n x n . 

Then (p is a linear functional on F” . 

• Define < p : PCR) -> R by cp(p) — 'ip" (5) + lp(A). Then cp is a linear 
functional on "PIR). 

• Define (p: V(R) — >■ R by cp(p) — Jq p{x) dx. Then (p is a linear 
functional on ThR). 

The vector space C(V, F) also gets a special name and special notation: 
3.94 Definition dual space, V' 

The dual space of V, denoted V\ is the vector space of all linear 
functionals on V. In other words, V = C(V, F). 


3.F 


3.95 dim V’ = dim V 

Suppose V is finite-dimensional. Then V' is also finite-dimensional and 
dim V' = dim V. 


Proof This result follows from 3.61. 
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In the following definition, 3.5 implies that each cpj is well defined. 


3.96 Definition dual basis 


If vi v n is a basis of V, then the dual basis of vi , . . . , v n is the list 

(pi cp n of elements of V. where each q>j is the linear - functional on V 

such that 


<Pj(vk) 


1 if k = j, 
0 if k ± j. 


3.97 Example What is the dual basis of the standard basis e\,...,e n 
of F”? 


Solution For 1 < j <n, define cpj to be the linear functional on F” that 
selects the j th coordinate of a vector in F”. In other words, 


<Pj(x 1 X n ) = Xj 


for (x !,..., x n ) SF". Clearly 

<Pj(e k ) 


j 1 if k = j, 
(0 if k ± j. 


Thus (pi, ... ,cp n is the dual basis of the standard basis e \ , . . . , e n of F” . 


The next result shows that the dual basis is indeed a basis. Thus the 
terminology “dual basis” is justified. 

3.98 Dual basis is a basis of the dual space 

Suppose V is finite-dimensional. Then the dual basis of a basis of V is a 
basis of V'. 

Proof Suppose vi, . . . , v n is a basis of V. Let (pi (p n denote the dual 

basis. 

To show that <pi , . . . , (p n is a linearly independent list of elements of V', 
suppose <3 1 a n e F are such that 

ai(pi H F a n (p n = 0 . 

Now + • • • + a n (p n ){vj ) = aj for j — 1 The equation 

above thus shows that ai =••• = «„ = 0. Hence cpi, ... ,(p n is linearly 
independent. 

Now 2.39 and 3.95 imply that (pi , . . . , (p n is a basis of V'. ■ 
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In the definition below, note that if T is a linear map from V to W then T' 
is a linear map from W' to V. 

3.99 Definition dual map, T' 

If T € C( V, W ), then the dual map of T is the linear map T' € C( W' , V') 
defined by T'((p) = (p o T for cp € W. 

If T e C(V,W) and <p € IV', then T'(cp) is defined above to be the 
composition of the linear maps cp and T. Thus T'(cp) is indeed a linear map 
from V to F; in other words, T'((p) e V. 

The verification that T' is a linear map from W' to V is easy: 

• If (p, x/r e W', then 

T'(sp + f) = (cp + f) o T — cpoT + xfr oT — T'(cp) + T'( f). 

• If A e F and <p e W', then 

T\ktp) = (A <p) o T — X(cp o T) — A T'{<p). 

In the next example, the prime notation is used with two unrelated mean- 
ings: D' denotes the dual of a linear map D, and p' denotes the derivative of 
a polynomial p. 


3.100 Example Define D : TTR) — »■ P(R) by Dp = p'. 

• Suppose ip is the linear functional on PfR) defined by (p(p) = p( 3). 
Then D'(cp) is the linear functional on PfR) given by 

( D\(p)){p ) = (<p o D)(p) = (p(Dp) = <p(p') = p'{ 3). 

In other words, D'(cp) is the linear functional on 'P(R) that takes p to 

P\ 3). 

• Suppose <p is the linear functional on 'P(R) defined by <p(p) = / () ' p. 
Then D'{cp) is the linear functional on T^R) given by 

( D'((p)){p ) = ((poD)(p) = <p{Dp ) = (p(p') = f p = p(l)-piO). 

Jo 

In other words, D'((p) is the linear functional on ThR) that takes p to 

p{V-p{ 0)- 
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The first two bullet points in the result below imply that the function that 
takes T to T' is a linear map from C(V, W) to C(W ' , V'). 

In the third bullet point below, note the reversal of order from S T on the 
left to T' S' on the right (here we assume that U is a vector space over F). 

3.101 Algebraic properties of dual maps 

• (S + T)' = S' + T' for all S,T € C(V, W). 

• (XT)' = XT' for all A e F and all T e C(V, W). 

• (ST)' = T'S' for all T e C(U, V) and all S e C(V, W ). 

Proof The proofs of the first two bullet points above are left to the reader. 
To prove the third bullet point, suppose cp e W'. Then 

(ST)'(cp) = cpo(ST) = (<poS)oT = T'(<poS) = T'(S'(<p)) = (T'S’)(<p), 

where the first, third, and fourth equal- 
ities above hold because of the defini- 
tion of the dual map, the second equality 
holds because composition of functions 
is associative, and the last equality fol- 
lows from the definition of composition. 

The equality of the first and last 
terms above for all <p e W’ means that 
(ST)' = T'S'. m 

The Null Space and Range of the Dual of a Linear Map 

Our goal in this subsection is to describe null T' and range T' in terms of 
range T and null T. To do this, we will need the following definition. 

3.102 Definition annihilator, U° 

For U C V, the annihilator of U, denoted U°, is defined by 
U° = {<p e V' : <p(u ) = 0 for all u e U}. 


Some books use the notation V* 
and T* for duality instead of V' 
and T'. However, here we reserve 
the notation T* for the adjoint, 
which will be introduced when we 
study linear maps on inner product 
spaces in Chapter 7. 


3.103 Example Suppose U is the subspace of T^R) consisting of all 
polynomial multiples of x 2 . If tp is the linear functional on 'P(R) defined by 
(pip) = p'( 0), then (p e U°. 
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For U C V, the annihilator U° is a subset of the dual space V. Thus U° 
depends on the vector space containing U, so a notation such as Uy would be 
more precise. However, the containing vector space will always be clear from 
the context, so we will use the simpler notation U°. 

3.104 Example Let ei, C2, e3,C4, es denote the standard basis of R 5 , and 
let (pi,(p2,<P3,<P4, <P5 denote the dual basis of (R 5 ) . Suppose 

U = span(ei, C2) = {(*1, *2. 0 , 0 , 0 ) e R 5 : xi, X2 e Rj. 

Show that U° — span(^3, ^4, cps ). 

Solution Recall (see 3.97) that < pj is the linear functional on R 5 that selects 
that j th coordinate: (p j (xi , X2 , X3 , X4 , X5) = xj . 

First suppose (p e span(<£>3, (p4. cps). Then there exist C3, C4, C5 e R such 
that (p = c 3 (p 3 + C4<p4 + c^cps- If (xi , X2, 0, 0, 0) € U , then 

<p(x 1,X2, 0,0,0) = {c 3 (p 3 + C4^4 + C5<Ps)(Xl , X2, 0, 0, 0) = 0. 

Thus <p e U°. In other words, we have shown that span (993, <^4. (ps) c U°. 

To show the inclusion in the other direction, suppose cp € U°. Because 
the dual basis is a basis of (R 5 ), there exist ci, C2, C3, C4, C5 e R such that 
(p = c\(p\ + C2<P2 + c 3 (p 3 + c^q)^ + cscps- Because e\ e U and <p e U°, we 
have 


0 = cp(e 1 ) = (ci(pi + C 2 (p2 + c 3 (P 3 + c 4 (p 4 + c 5 cp 5 )(ei) = c\. 

Similarly, e2 € U and thus C2 — 0. Hence 1 p = c 3 cp 3 + C4(p4 + cscps. Thus 
(p € span(^3, q>4, (p 3 ), which shows that U° C span(^3, cp4, (ps). 


3.105 The annihilator is a subspace 
Suppose U C V. Then U° is a subspace of V'. 


Proof Clearly 0 e U° (here 0 is the zero linear functional on V ), because 
the zero linear functional applied to every vector in U is 0. 

Suppose (p,\fr g U°. Thus <p, x/z e V' and <p(u ) = t / t ( m ) = 0 for every 
u s U. If u e U, then (cp + 1 j/){u) — cp(u) + i/r(w) = 0 + 0 = 0. Thus 
(p + \j/ € U°. 

Similarly, U° is closed under scalar multiplication. Thus 1.34 implies that 
U° is a subspace of V'. ■ 
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The next result shows that dim U° is the difference of dim V and dim U. 
For example, this shows that if U is a 2-dimensional subspace of R 5 , then U° 
is a 3-dimensional subspace of (R 5 ), as in Example 3.104. 

The next result can be proved following the pattern of Example 3.104: 
choose a basis u i ..... u m of U, extend to a basis u i , . . . , u m , . . . , u n of V, 

let (pi (p m (p„ be the dual basis of V', and then show (p m +i ,<p n 

is a basis of U°, which implies the desired result. 

You should construct the proof outlined in the paragraph above, even 
though a slicker proof is presented here. 

3.106 Dimension of the annihilator 

Suppose V is finite-dimensional and U is a subspace of V. Then 
dim U + dim U° = dim V. 

Proof Let i e £(£/, V) be the inclusion map defined by i ( u ) = u for u € U. 
Thus /' is a linear map from V to U'. The Fundamental Theorem of Linear 
Maps (3.22) applied to i' shows that 

dim range /' + dim null /' = dim V'. 

However, null i' = U° (as can be seen by thinking about the definitions) and 
dim V' = dim V (by 3.95), so we can rewrite the equation above as 

dim range i' + dim U° = dim V. 

If cp € U\ then (p can be extended to a linear functional t// on V (see, 
for example, Exercise 11 in Section 3. A). The definition of i' shows that 
/'( r/r) = (p. Thus i p € range/', which implies that range/' = U'. Hence 
dim range /' = dim U' = dim U, and the displayed equation above becomes 
the desired result. ■ 

The proof of part (a) of the result below does not use the hypothesis that 
V and W are finite-dimensional. 

3.107 The null space of T' 

Suppose V and W are finite-dimensional and T e C{ V. W). Then 

(a) null T' = (range T)°; 

(b) dim null T’ = dim null T + dim W — dim V. 
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Proof 

(a) First suppose (p e null T'. Thus 0 = T'{cp) — <p o T. Hence 

0 — {(p o D(v) = <p(Tv) for every v e V. 

Thus cp e (range T )°. This implies that null T' c (range T)° . 

To prove the inclusion in the opposite direction, now suppose that 
(p € (range T) 0 . Thus (p(Tv) — 0 for every vector v e V. Hence 
0 = (p o T — T'(<p). In other words, i p e null T\ which shows that 
(range T)° C null T\ completing the proof of (a). 

(b) We have 

dim null T' = dim(range T)° 

= dim W — dim range T 
= dim W — (dim V — dim null T ) 

= dim null T + dim W — dim V, 

where the first equality comes from (a), the second equality comes from 
3.106, and the third equality comes from the Fundamental Theorem of 
Linear Maps (3.22). ■ 

The next result can be useful because sometimes it is easier to verify that 
T' is injective than to show directly that T is surjective. 

3.108 T surjective is equivalent to T' injective 

Suppose V and W are finite-dimensional and T € £{V, W). Then T is 
surjective if and only if T' is injective. 

Proof The map T e £( V. W) is surjective if and only if range T — W, 
which happens if and only if (range T)° = {0}, which happens if and only if 
null T' = {0} [by 3.107(a)], which happens if and only if T' is injective. ■ 

3.109 The range of T' 

Suppose V and W are finite-dimensional and T e C{V, W). Then 

(a) dim range T' = dim range T ; 

(b) range T' = (null T)°. 
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Proof 

(a) We have 


dim range T' — dim W' — dim null T' 

— dim W — dim(range T)° 

— dim range T, 

where the first equality comes from the Fundamental Theorem of Linear 
Maps (3.22), the second equality comes from 3.95 and 3.107(a), and 
the third equality comes from 3.106. 

(b) First suppose (p e range T'. Thus there exists r// g W such that 
<p = T'(i/r). If v g null T, then 

(p{v) — (T\\ff))v = (fo T)(v) = t fr(Tv) = t/r(0) = 0. 

Hence i tp 6 (null T)°. This implies that range T' C (null T)°. 

We will complete the proof by showing that range T' and (null T)° 
have the same dimension. To do this, note that 

dim range T' — dim range T 

= dim V — dim null T 
= dim (null T)°, 

where the first equality comes from (a), the second equality comes from 
the Fundamental Theorem of Linear Maps (3.22), and the third equality 
comes from 3. 106. ■ 

The next result should be compared to 3.108. 

3.1 io T injective is equivalent to T' surjective 

Suppose V and W are finite-dimensional and T e C(V, W ) . Then T is 
injective if and only if T' is surjective. 

Proof The map T e £(K, W) is injective if and only if null T = {0}, 
which happens if and only if (null T)° — V', which happens if and only if 
range T' — V' [by 3.109(b)], which happens if and only if T' is surjective. ■ 
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The Matrix of the Dual of a Linear Map 

We now define the transpose of a matrix. 

3.111 Definition transpose, A 1 

The transpose of a matrix A, denoted A\ is the matrix obtained from 
A by interchanging the rows and columns. More specifically, if A is an 
m-by-n matrix, then A 1 is the n-by-m matrix whose entries are given by 
the equation 

(A l )k,j = A jk . 


( 5 

3.112 Example If ,4 = 3 8 

V -4 2 

Note that here A is a 3-by-2 matrix and A 1 is a 2-by-3 matrix. 

The transpose has nice algebraic properties: (A + C) 1 = A 1 + C l and 
(XAf = A A 1 for all m-by-n matrices A. C and all A € F (see Exercise 33). 

The next result shows that the transpose of the product of two matrices is 
the product of the transposes in the opposite order. 

3.113 The transpose of the product of matrices 
If A is an m-by-n matrix and C is an n-by-p matrix, then 

(AC) 1 = CM 1 . 

Proof Suppose 1 < k < p and 1 < j < m. Then 
(UC)') kJ = ( AC ) u 

n 

- 'Yh A J, rC r,k 

r= 1 
n 

= £( c %A A %j 

r= 1 

= (C'A'hj. 



Thus (^4C) £ = CM 1 , as desired. 
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The setting for the next result is the assumption that we have a basis 
vi , . . . , v n of V, along with its dual basis cp\, ... ,<p n of V. We also have a 
basis w i , . . . , w m of W, along with its dual basis i/q , . . . , \j/ m of W'. Thus 
A4(T) is computed with respect to the bases just mentioned of V and W, 
and M{T') is computed with respect to the dual bases just mentioned of W' 
and V'. 

3.1 14 The matrix of T' is the transpose of the matrix of T 
Suppose T € £(V, W). Then M(T') = (_A/f(T)) 1 . 

Proof Let A = M.(T) and C = Suppose 1 < j < m and 

l < k < n. 

From the definition of JA(T') we have 

n 

T (ijjj ) = y ] C r ,j(pr- 

r= 1 

The left side of the equation above equals i j/j o T. Thus applying both sides 
of the equation above to \\ gives 

n 

(xjrj O T)(v k ) = ^2 C rJ<Pr(v k ) 

r= 1 

= c kJ . 

We also have 

(fj ° T)(v, t) = f j(Tv k ) 

m 

= i'j( J y2 A r, k Wr^) 

r= 1 
m 

= XI A r,k^j(Wr) 
r= 1 

= A j,k- 

Comparing the last line of the last two sets of equations, we have C k j = A ,\ k - 
Thus C = A l . In other words, AA(T') = (Adfr)) 1 , as desired. ■ 
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The Rank of a Matrix 

We begin by defining two nonnegative integers that are associated with each 
matrix. 


3.115 Definition row rank, column rank 
Suppose A is an m-by-n matrix with entries in F. 

• The row rank of A is the dimension of the span of the rows of A in 
F 1 ’". 

• The column rank of A is the dimension of the span of the columns 
of A in F m ’ 1 . 


3.116 Example Suppose A — 
and the column rank of A. 


f 4 7 1 8 
v 3 5 2 9 


Find the row rank of A 


Solution The row rank of A is the dimension of 

span(( 4 7 1 8), (3 5 2 9)) 

in F 1,4 . Neither of the two vectors listed above in F 1,4 is a scalar multiple 
of the other. Thus the span of this list of length 2 has dimension 2. In other 
words, the row rank of A is 2. 

The column rank of A is the dimension of 



in F 2,1 . Neither of the first two vectors listed above in F 2,1 is a scalar multiple 
of the other. Thus the span of this list of length 4 has dimension at least 2. 
The span of this list of vectors in F 2,1 cannot have dimension larger than 2 
because dimF 2,1 = 2. Thus the span of this list has dimension 2. In other 
words, the column rank of A is 2. 


Notice that no bases are in sight in the statement of the next result. Al- 
though M.{T) in the next result depends on a choice of bases of V and W, 
the next result shows that the column rank of M{T) is the same for all such 
choices (because range T does not depend on a choice of basis). 
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3.1 17 Dimension of range T equals column rank of M(T) 

Suppose V and W are finite-dimensional and T e C(V, W). Then 
dimrange T equals the column rank of M.(T). 

Proof Suppose rq v n is a basis of V and vtq , . . . , w m is a basis of W. The 

function that takes w e span(T vq , ... ,T v n ) to M.(w) is easily seen to be an 
isomorphism from span(T rq , ,Tv n ) onto span(.A/f(7Tq ), . ... A4(Tv n )). 

Thus dimspan(7Tq Tv n ) = dim span(Al(Tvi), ... ,Al(7’v„)), where 

the last dimension equals the column rank of A i(T). 

It is easy to see that range T — span(7Tq, . . . , Tv n ). Thus we have 

dimrangeT = dimspan(7Tq Tv n ) — the column rank of M.{T), as 

desired. ■ 

In Example 3.116, the row rank and column rank turned out to equal each 
other. The next result shows that this always happens. 

3.118 Row rank equals column rank 

Suppose A € Y m ' n . Then the row rank of A equals the column rank of A. 

Proof Define T : F”’ 1 — >• F" 7,1 by T x = Ax. Thus M{T) = A, where 
M(T) is computed with respect to the standard bases of F”’ 1 and F m ' 1 . Now 

column rank of A — column rank of M( T) 

— dim range T 

— dimrange T' 

= column rank of A4(T r ) 

— column rank of A 1 

— row rank of A. 

where the second equality above comes from 3.117, the third equality comes 
from 3.109(a), the fourth equality comes from 3.1 17 (where Ai(T') is com- 
puted with respect to the dual bases of the standard bases), the fifth equality 
comes from 3.114, and the last equality follows easily from the definitions. ■ 

The last result allows us to dispense with the terms “row rank” and “column 
rank” and just use the simpler term “rank”. 

3.119 Definition rank 

The rank of a matrix A e F m,n is the column rank of A. 
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EXERCISES 3.F 


1 Explain why every linear functional is either surjective or the zero map. 

2 Give three distinct examples of linear functionals onR[ Q,1 l 

3 Suppose V is finite-dimensional and v € V with v ^ 0. Prove that there 
exists (p € V' such that <p(v) = 1. 

4 Suppose V is finite-dimensional and U is a subspace of V such that 
U / V. Prove that there exists <p € V’ such that <p(u) — 0 for every 
u e U but <p ^ 0. 

5 Suppose V\ , . . . , V m are vector spaces. Prove that (hj x ■ ■ ■ x V m )’ and 
V\ x ■ ■ ■ x V m ' arc isomorphic vector spaces. 

6 Suppose V is finite-dimensional and vi v m e V. Define a linear 

map T : V' F m by 

F(<P) = (<jo(vi <p(v m )). 


(a) Prove that v\, ... ,v m spans V if and only if T is injective. 

(b) Prove that v\, ... ,v m is linearly independent if and only if T is 
surjective. 

7 Suppose m is a positive integer. Show that the dual basis of the basis 

l,x,...,x m ofTVRjis (po.cpi (p m , where <pj (p) = p J j ^ ) . Here 

denotes the j th derivative of p, with the understanding that the 0 th 
derivative of p is p. 

8 Suppose m is a positive integer. 

(a) Show that 1, x — 5, . . . , (x — 5) m is a basis of V m (R). 

(b) What is the dual basis of the basis in part (a)? 

9 Suppose vi v n is a basis of V and (p\, . . . ,(p n is the corresponding 

dual basis of V'. Suppose \f/ € V'. Prove that 

if = f{v\)(pi -\ k i ir(v n )<p n . 


10 Prove the first two bullet points in 3. 101 . 
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11 Suppose A is an m-by-n matrix with A ^ 0. Prove that the rank of A 
is 1 if and only if there exist (c\, . . . ,c m ) e F m and (d\, . . . , d n ) e F" 
such that A jj( = Cjd ^ for every j = 1 , ,m and every k = 1 ..... /7 . 

12 Show that the dual map of the identity map on V is the identity map 
on V'. 

13 Define T : R 3 — ► R 2 by T(x, y, z ) = (Ax + 5 y + 6 z, lx + 8y + 9 z). 
Suppose (pi , <p 2 denotes the dual basis of the standard basis of R 2 and 
^ 1 ,^ 2 , 1^3 denotes the dual basis of the standard basis of R 3 . 

(a) Describe the linear functionals T'((pi) and T'(cp 2 ). 

(b) Write T'(cpi) and T'(cp 2 ) as linear combinations of i jr\, r/r 2 , 1/73. 

14 Define T : V(R) —>■ ^(R) by ( Tp)(x ) = x 2 p(x) + p" (x) for x 6 R. 

(a) Suppose (p € T^fR)' is defined by (p(p) = p'( 4). Describe the 
linear functional T'((p) on P(R). 

(b) Suppose (p € ^(R) 7 is defined by (p(p) = f 0 p(x) dx. Evaluate 
(T\(p))(x 3 ). 

15 Suppose W is finite-dimensional and T e C(V. W). Prove that T' — 0 
if and only if T = 0. 

16 Suppose V and W are finite-dimensional. Prove that the map that takes 
T e C(V, W) to T' s £(W', V ') is an isomorphism of C(V, W) onto 
C(W, V r ). 

17 Suppose U C V. Explain why U° = {<p e V' : U C null<p}. 

18 Suppose V is finite-dimensional and U C V. Show that U — {0} if and 
only if U° = V'. 

19 Suppose V is finite-dimensional and U is a subspace of V. Show that 
U = V if and only if U° = {0}. 

20 Suppose U and W are subsets of V with U C W. Prove that ll /0 c U°. 

21 Suppose V is finite-dimensional and U and W are subspaces of V with 
W° C U°. Prove that U C W. 

22 Suppose U, W are subspaces of V. Show that (U + W)° = U° n W°. 
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23 Suppose V is finite-dimensional and U and W are subspaces of V. Prove 
that (U n W)° = U° + W°. 

24 Prove 3.106 using the ideas sketched in the discussion before the state- 
ment of 3.106. 

25 Suppose V is finite-dimensional and U is a subspace of V. Show that 

U — {v 6 V : <p(v) — 0 for every <p e U 0 }. 

26 Suppose V is finite-dimensional and F is a subspace of V. Show that 

T = {v e V : (p{v) = 0 for every i p e T}°. 

27 Suppose T e C(Vs(R.), PsCR)) and null T' — span(^), where cp is 
the linear functional on F’s(R) defined by (p( p) — p( 8). Prove that 
range T = {p € Vs(R) : p{ 8) = 0}. 

28 Suppose V and W are finite-dimensional, T e C{V, W), and there exists 
cp € W' such that null T' = span(ip). Prove that range T = null i p. 

29 Suppose V and W are finite-dimensional, T e C{V, W), and there exists 
<p e V' such that range T' — span(^). Prove that null T = null cp. 

30 Suppose V is finite-dimensional and cpi , . . . , cp m is a linearly independent 
list in V'. Prove that 

dim((null <p\) D ■ ■ ■ FI (null cp m )) = (dim V) — m. 

31 Suppose V is finite-dimensional and cpi , . . . , cp n is a basis of V'. Show 
that there exists a basis of V whose dual basis is (pi , . . . , <p n . 

32 Suppose T € C(V), and u \ , . . . , u n and vi , . . . , v„ are bases of V. Prove 
that the following are equivalent: 

(a) T is invertible. 

(b) The columns of M{T) are linearly independent in F”’ 1 . 

(c) The columns of M. (T) span F"’ 1 . 

(d) The rows of M.(T) are linearly independent in F 1,n . 

(e) The rows of M.(T) span F 1 ’”. 

Here M(T ) means M(T, (u i, . . . , u n ), (vi, . . . , u n )). 
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33 Suppose m and n are positive integers. Prove that the function that takes 
A to A 1 is a linear map from F m ,n to F” ,m . Furthermore, prove that this 
lineal - map is invertible. 

34 The double dual space of V, denoted V", is defined to be the dual space 
of V'. In other words, V" = {V')'. Define A : V -»• V" by 

(Av)(<p) = <p{v) 

for v e V and cp e V'. 

(a) Show that A is a linear map from V to V". 

(b) Show that if T e C(V), then T" o A = A o T, where T" = (T r )'. 

(c) Show that if V is finite-dimensional, then A is an isomorphism 
from V onto V". 

\ Suppose V is finite -dimensional. Then V and V' are isomorphic, but 
finding an isomorphism from V onto V' generally requires choosing a 
basis ofV. In contrast, the isomorphism A from V onto V" does not 
require a choice of basis and thus is considered more natural .] 

35 Show that (T’(R)) / and R°° are isomorphic. 

36 Suppose U is a subspace of V. Let i : U V be the inclusion map 
defined by i(u) = u. Thus i' e L{V ' , U'). 

(a) Show that null/' = [7°. 

(b) Prove that if V is finite-dimensional, then range i' = U'. 

(c) Prove that if V is finite-dimensional, then /' is an isomorphism 
from V'/U° onto U'. 

[The isomorphism in part (c) is natural in that it does not depend on a 
choice of basis in either vector space.] 

37 Suppose U is a subspace of V. Let re : V — V / U be the usual quotient 
map. Thus n' e C((V/U)', V'). 

(a) Show that n' is injective. 

(b) Show that range n' - U°. 

(c) Conclude that n' is an isomorphism from (V/U)' onto U°. 

[The isomorphism in part (c) is natural in that it does not depend on a 
choice of basis in either vector space. In fact, there is no assumption 
here that any of these vector spaces are finite-dimensional] 
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Polynomials 


This short chapter contains material on polynomials that we will need to 
understand operators. Many of the results in this chapter will already be 
familiar to you from other courses; they are included here for completeness. 

Because this chapter is not about linear algebra, your instructor may go 
through it rapidly. You may not be asked to scrutinize all the proofs. Make 
sure, however, that you at least read and understand the statements of all the 
results in this chapter — they will be used in later chapters. 

The standing assumption we need for this chapter is as follows: 

4.1 Notation F 

F denotes R or C. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ Division Algorithm for Polynomials 

■ factorization of polynomials over C 

■ factorization of polynomials over R 
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Complex Conjugate and Absolute Value 

Before discussing polynomials with complex or real coefficients, we need to 
learn a bit more about the complex numbers. 

4.2 Definition Rez, Im: 

Suppose z = a + bi , where a and b are real numbers. 

• The real part of z, denoted Re z, is defined by Re z = a. 

• The imaginary part of z, denoted Imz, is defined by Imz = b. 

Thus for every complex number z, we have 
z = Rez + (Im z)i. 

4.3 Definition complex conjugate, z, absolute value, \z\ 

Suppose z € C. 

• The complex conjugate of zeC, denoted z, is defined by 

z = Rez — (Imz)/. 

• The absolute value of a complex number z, denoted \z\, is defined 

by 

|z| = yj (Rez) 2 + (Imz) 2 . 


4.4 Example Suppose z = 3 + 2i. Then 

• Re z = 3 and Im z = 2; 

• z = 3 — 2/ ; 

• \z\ = V3 2 + 2 2 = x/l3. 

Note that |z| is a nonnegative number for every z e C. 

The real and imaginary parts, com- 
plex conjugate, and absolute value have 
the following properties: 


You should verify that z = z if and 
only if z is a real number. 
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4.5 Properties of complex numbers 

Suppose w, z € C. Then 

sum of z and z 

z + z = 2 Re z; 

difference of z and z 

z — z = 2(Imz)z; 

product of z and z 

i i2 

zz = |z|; 

additivity and multiplicativity of complex conjugate 

w + z = w + z and vvz = vvz; 

conjugate of conjugate 

z = z; 

real and imaginary parts are bounded by |z| 

| Re z | < | z | and | Im z | < | z | 

absolute value of the complex conjugate 

|z| = | z | ; 

multiplicativity of absolute value 

|wz| = | w| | z | ; 

Triangle Inequality 

|vv + z | < |w| + |z|. 

Proof Except for the last item, the routine verifications of the assertions 
above are left to the reader. To verify the last item, we have 

| w + z| 2 = (w + z)(iv + z) 

= wvv + zz + wz + zvv 
= |w| 2 + |z| 2 + vvz + wz 
= |w| 2 + |z| 2 + 2Re(wz) 

< \w\ 2 + |z| 2 + 2|wz| 

= \w\ 2 + |z| 2 + 2|w| |z| 

= (k| + \z\) 2 . 

Taking the square root of both sides of the inequality \w + z I 2 < (H + |z|) 2 
now gives the desired inequality. ■ 
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Uniqueness of Coefficients for Polynomials 

Recall that a function p : F — >• F is called a polynomial with coefficients in F 
if there exist ao, . . . , a m € F such that 

4.6 p(z) = ao + ci\z + a 2 z 2 4 b a m z m 

for all z € F. 

4.7 If a polynomial is the zero function, then all coefficients are 0 

Suppose ao ,a m e F. If 


ao ~b a\z + • • ■ -f a m z m — 0 
for every z € F, then a 0 = ■ ■ ■ = a m =0. 

Proof We will prove the contrapositive. If not all the coefficients are 0, then 
by changing m we can assume a m ^ 0. Let 

l^ol + W \ I + • • • + Wm-i I , . 

z = : : b 1. 

\(lm | 

Note that z > 1, and thus z 7 < z m ~ x for j = 0, 1, . . . , m — 1. Using the 
Triangle Inequality, we have 

|< 2 o + < 2 iz + • • • + a m ~\z m 1 |<(|rzo| + |rti| + ''' + \a m -\ \)z m 1 

< \a m z m \. 

Thus ao + a\z + ■•• + a m -iz m ~ 1 ^ —a m z m . Hence we conclude that 
a 0 + aiz-\ b a m -\z m ~ 1 + a m z m ^ 0. ■ 

The result above implies that the coefficients of a polynomial are uniquely 
determined (because if a polynomial had two different sets of coefficients, 
then subtracting the two representations of the polynomial would give a 
contradiction to the result above). 

Recall that if a polynomial p can be written in the form 4.6 with a m yb 0, 
then we say that p has degree m and we write deg p = m. 

The degree of the 0 polynomial is 
defined to be — oo. When necessary, use 
the obvious arithmetic with — oo. For 
example, — oo < m and — oo + m — 
— oo for every integer m. 


The 0 polynomial is declared to 
have degree — oo so that excep- 
tions are not needed for various 
reasonable results. For example, 
deg (pq) = deg p + deg q even if 

P = o. 
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The Division Algorithm for Polynomials 

If p and 5 are nonnegative integers, with s ^ 0, then there exist nonnegative 
integers q and r such that 

p — sq + r 

and r < s. Think of dividing p by s, getting quotient q with remainder r. Our 
next task is to prove an analogous result for polynomials. 

The result below is often called the 
Division Algorithm for Polynomials, al- 
though as stated here it is not really an 
algorithm, just a useful result. 

Recall that 'P(F) denotes the vector space of all polynomials with co- 
efficients in F and that V m (F) is the subspace of 'P(F) consisting of the 
polynomials with coefficients in F and degree at most m. 

The next result can be proved without linear algebra, but the proof given 
here using linear algebra is appropriate for a linear algebra textbook. 

4.8 Division Algorithm for Polynomials 

Suppose that p,s e V(F), with ,s - ^ 0. Then there exist unique 
polynomials q, r e V(F) such that 

p = sq + r 


Think of the Division Algorithm for 
Polynomials as giving the remain- 
der r when p is divided by s. 


and degr < deg s. 

Proof Let n = deg p and m = deg s. If n < m, then take q = 0 and r — p 
to get the desired result. Thus we can assume that n > m. 

Define T : V n - m (F) x V m -i(F) -> V n (F) by 

T(q,r) = sq + r. 

The reader can easily verify that T is a linear map. If (q, r) e null T, then 
sq + r — 0, which implies that q — 0 and r — 0 [because otherwise 
deg sq > m and thus sq cannot equal — r]. Thus dim null T — 0 (proving the 
“unique” part of the result). 

From 3.76 we have 

dim(P„_ m (F) x V m -i(Y)) = (n - m + 1) + (m - 1 + 1) = n + 1. 

The Fundamental Theorem of Linear Maps (3.22) and the equation displayed 
above now imply that dim range T = n + 1, which equals dim V n (¥). Thus 
range T = V n (F ) , and hence there exist q e V n ~ m (F) and r e V m -\ (F) 
such that p — T(q, r) — sq + r. ■ 
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Zeros of Polynomials 

The solutions to the equation p(z) — 0 play a crucial role in the study of a 
polynomial p e V(F). Thus these solutions have a special name. 

4.9 Definition zero of a polynomial 

A number A e F is called a zero (or root) of a polynomial p € V(F) if 

p(X) = 0. 


4.10 Definition factor 

A polynomial s € V(F) is called a factor of p e V(F) if there exists a 
polynomial q € V(F) such that p = sq. 

We begin by showing that A is a zero of a polynomial p € V{ F) if and 
only if z — A is a factor of p. 

4.1 1 Each zero of a polynomial corresponds to a degree-1 factor 

Suppose p € V(F) and A e F. Then p( A) = 0 if and only if there is a 
polynomial q € V{F) such that 

p(z) = (z — X)q(z) 


for every z € F. 

Proof One direction is obvious. Namely, suppose there is a polynomial 
q e 'P(F) such that p(z) = (z — A )q(z) for all z e F. Then 

p{ A) = (A — A)<?(A) = 0, 

as desired. 

To prove the other direction, suppose p( A) = 0. The polynomial z — A 
has degree 1. Because a polynomial with degree less than 1 is a constant 
function, the Division Algorithm for Polynomials (4.8) implies that there exist 
a polynomial q € V(F) and a number r e F such that 

p{z) = (z - A )q(z) + r 

for every z e F. The equation above and the equation p{ A) = 0 imply that 
r = 0. Thus p(z) = (z — X)q(z) for every z e F. ■ 

Now we can prove that polynomials do not have too many zeros. 
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4.12 A polynomial has at most as many zeros as its degree 

Suppose p € 'P(F) is a polynomial with degree m > 0. Then p has at 
most m distinct zeros in F. 

Proof If m — 0, then p(z) = ao ^ 0 and so p has no zeros. 

If m — 1, then p(z) — a 0 + aqz, with a\ ^ 0, and thus p has exactly 
one zero, namely, — cio/ai. 

Now suppose m > 1. We use induction on m, assuming that every 
polynomial with degree m — 1 has at most m — 1 distinct zeros. If p has no 
zeros in F, then we are done. If p has a zero A e F, then by 4. 1 1 there is a 
polynomial q such that 


p(z) = (z - X)q(z) 

for all z e F. Clearly deg q — m — 1 . The equation above shows that if 
p(z) — 0, then either z = A or q(z) = 0. In other words, the zeros of p 
consist of A and the zeros of q. By our induction hypothesis, q has at most 
m — 1 distinct zeros in F. Thus p has at most m distinct zeros in F. ■ 

Factorization of Polynomials over C 

So far we have been handling polynomials with complex coefficients and 
polynomials with real coefficients simultaneously through our convention that 
F denotes R or C. Now we will see some differences between these two cases. 
First we treat polynomials with complex coefficients. Then we will use our 
results about polynomials with complex coefficients to prove corresponding 
results for polynomials with real coefficients. 

The next result, although called the 
Fundamental Theorem of Algebra, uses 
analysis its proof. The short proof pre- 
sented here uses tools from complex 
analysis. If you have not had a course in 
complex analysis, this proof will almost 
certainly be meaningless to you. In that 
case, just accept the Fundamental The- 
orem of Algebra as something that we 
need to use but whose proof requires 
more advanced tools that you may learn 
in later courses. 


The Fundamental Theorem of Al- 
gebra is an existence theorem. Its 
proof does not lead to a method for 
finding zeros. The quadratic for- 
mula gives the zeros explicitly for 
polynomials of degree 2. Similar 
but more complicated formulas ex- 
ist for polynomials of degree 3 and 
4. No such formulas exist for poly- 
nomials of degree 5 and above. 
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4.13 Fundamental Theorem of Algebra 

Every nonconstant polynomial with complex coefficients has a zero. 

Proof Let p be a nonconstant polynomial with complex coefficients. Sup- 
pose p has no zeros. Then \ / p is an analytic function on C. Furthermore, 
\p(z)\ — > oo as \z\ — »■ oo, which implies that l/p — >■ 0 as |z| — oo. Thus 
1 //? is a bounded analytic function on C. By Liouville’s theorem, every such 
function is constant. But if l/p is constant, then p is constant, contradicting 
our assumption that p is nonconstant. ■ 

Although the proof given above is probably the shortest proof of the 
Fundamental Theorem of Algebra, a web search can lead you to several other 
proofs that use different techniques. All proofs of the Fundamental Theorem 
of Algebra need to use some analysis, because the result is not true if C is 
replaced, for example, with the set of numbers of the form c + di where c, d 
are rational numbers. 

Remarkably, mathematicians have 
proved that no formula exists for the ze- 
ros of polynomials of degree 5 or higher. 
But computers and calculators can use 
clever numerical methods to find good 
approximations to the zeros of any poly- 
nomial, even when exact zeros cannot 
be found. 

For example, no one will ever be 
able to give an exact formula for a zero 
of the polynomial p defined by 

p(x) = x 5 — 5x 4 — 6 x 3 + 17x 2 +4x— 7. 

However, a computer or symbolic cal- 
culator can find approximate zeros of 
this polynomial. 

The Fundamental Theorem of Alge- 
bra leads to the following factorization 
result for polynomials with complex co- 
efficients. Note that in this factorization, 
the numbers Ai, . . . , A m are precisely 
the zeros of p, for these are the only 
values of z for which the right side of 
the equation in the next result equals 0. 


The cubic formula, which was 
discovered in the 16 th century, 
is presented below for your 
amusement only. Do not memorize 
it. 

Suppose 

p(x) = ax 3 + bx 2 + cx + d, 

where a / 0. Set 

9 abc — 2 b 3 — 21a 2 d 
u = — 


and then set 


v = u + 


54 a 3 

/3 ac — b 2 \ 3 

V 9 a 2 ) ' 


9 a 2 

Suppose v > 0. Then 


b 

~Ya + 


yju + Vv + yju - Vv 

is a zero of p. 
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4.14 Factorization of a polynomial over C 

If p e V(C) is a nonconstant polynomial, then p has a unique factoriza- 
tion (except for the order of the factors) of the form 

p(z) = c(z — Ai) ■■■ (z — X m ), 

where c,X\,...,X m e C. 

Proof Let p € 'P(C) and let m — deg p. We will use induction on m. If 
m = 1 , then clearly the desired factorization exists and is unique. So assume 
that m > 1 and that the desired factorization exists and is unique for all 
polynomials of degree m — 1 . 

First we will show that the desired factorization of p exists. By the 
Fundamental Theorem of Algebra (4.13), p has a zero A. By 4.1 1, there is a 
polynomial q such that 


p{z) = (z — X)q (z) 

for all z € C. Because deg q — m — 1, our induction hypothesis implies that 
q has the desired factorization, which when plugged into the equation above 
gives the desired factorization of p. 

Now we turn to the question of uniqueness. Clearly c is uniquely deter- 
mined as the coefficient of z m in p. So we need only show that except for the 
order, there is only one way to choose A i X m . If 

(z - Ai) ■ ■ ■ (z - Xm) = (z - n) ■ ■ ■ (z - Zm) 

for all z e C, then because the left side of the equation above equals 0 when 
z = Ai, one of the t’s on the right side equals Ai. Relabeling, we can assume 
that ri = A i . Now for z ^ A i , we can divide both sides of the equation 
above by z — A i , getting 


(z - A 2 ) • • ■ (z - X m ) = (z - r 2 ) • • • (z - z m ) 


for all z e C except possibly z = A i . Actually the equation above holds 
for all z e C, because otherwise by subtracting the right side from the left 
side we would get a nonzero polynomial that has infinitely many zeros. The 
equation above and our induction hypothesis imply that except for the order, 
the A’s are the same as the r’s, completing the proof of uniqueness. ■ 
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Factorization of Polynomials over R 

A polynomial with real coefficients may 
have no real zeros. For example, the 
polynomial 1 + x 2 has no real zeros. 

To obtain a factorization theorem 
over R, we will use our factorization 
theorem over C. We begin with the fol- 
lowing result. 

4.15 Polynomials with real coefficients have zeros in pairs 

Suppose p € 'P(C) is a polynomial with real coefficients. If A e C is a 
zero of p, then so is A. 

Proof Let 

p(z) = ao + a\z -\ b a m z m , 

where a o a m are real numbers. Suppose A e C is a zero of p. Then 

flo T r?iA + • • • + — 0. 

Take the complex conjugate of both sides of this equation, obtaining 
ao + a\\ + a m X m — 0, 

where we have used basic properties of complex conjugation (see 4.5). The 
equation above shows that A is a zero of p. ■ 

We want a factorization theorem for 
polynomials with real coefficients. First 
we need to characterize the polynomi- 
als of degree 2 with real coefficients 
that can be written as the product of 
two polynomials of degree 1 with real 
coefficients. 

4.16 Factorization of a quadratic polynomial 

Suppose /),ceR. Then there is a polynomial factorization of the form 

x 2 + bx + c = (x — Ai)(x — A 2 ) 
with Ai, A 2 e R if and only if b 2 > 4c. 


Think about the connection be- 
tween the quadratic formula and 
4.16. 


The failure of the Fundamental 
Theorem of Algebra for R accounts 
for the differences between oper- 
ators on real and complex vector 
spaces, as we will see in later 
chapters. 
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Proof Notice that 


x 2 + bx + c 




First suppose b 2 < 4c. Then clearly 
the right side of the equation above is 
positive for every x € R. Flence the 
polynomial x 2 + bx + c has no real 
zeros and thus cannot be factored in the 
form (x — Ai)(x — A 2 ) with Ai, A 2 e R. 

Conversely, now suppose b 2 > 4c. Then there is a real number d such 
that d 2 — ^ — c. From the displayed equation above, we have 


The equation above is the basis 
of the technique called completing 
the square. 


x 2 + bx + c = 



which gives the desired factorization. 


The next result gives a factorization of a polynomial over R. The idea of 
the proof is to use the factorization 4.14 of p as a polynomial with complex 
coefficients. Complex but nonreal zeros of p come in pairs; see 4.15. Thus 
if the factorization of p as an element of V(C) includes terms of the form 
(x — A) with A a nonreal complex number, then (x — A) is also a term in the 
factorization. Multiplying together these two terms, we get 

(x 2 — 2(ReA)x + |A| 2 ), 

which is a quadratic term of the required form. 

The idea sketched in the paragraph above almost provides a proof of the 
existence of our desired factorization. Flowever, we need to be careful about 
one point. Suppose A is a nonreal complex number and (x — A) is a term in 
the factorization of p as an element of V(C). We are guaranteed by 4.15 that 
(x — A) also appears as a term in the factorization, but 4.15 does not state that 
these two factors appear the same number of times, as needed to make the 
idea above work. However, the proof works around this point. 

In the next result, either m or M may equal 0. The numbers A 1 A m 

are precisely the real zeros of p, for these are the only real values of x for 
which the right side of the equation in the next result equals 0. 
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4.17 Factorization of a polynomial over R 

Suppose p € 'P(R) is a nonconstant polynomial. Then p has a unique 
factorization (except for the order of the factors) of the form 

p{x) = c{x - Ai) • • • (x - X m )(x 2 + b\x + ci) • • • (x 2 + b M x + cm), 

where c, Ai, . . . , A m , b\ bM,c\, . . . ,cm G R, with bj 2 < 4 cj for 

each j . 

Proof Think of p as an element of V(C). If all the (complex) zeros of p are 
real, then we are done by 4.14. Thus suppose p has a zero A e C with A £ R. 
By 4.15, A is a zero of p. Thus we can write 

p(x) = (x — A)(x — A )q(x) 

= (x 2 — 2(ReA)x + |A| 2 )g(x) 

for some polynomial q € 'P(C) with degree two less than the degree of p. 
If we can prove that q has real coefficients, then by using induction on the 
degree of p, we can conclude that (x — A) appears in the factorization of p 
exactly as many times as (x — A). 

To prove that q has real coefficients, we solve the equation above for q, 
getting 

, , _ P(x) 

^ X x 2 — 2(ReA)x + |A| 2 

for all x € R. The equation above implies that q(x) e R for all x e R. 
Writing 

q(x) = ao + a\x + • • • + o n — 2 X n ~ 2 , 
where n — deg p and ao, ■ . . , a n - 2 £ C, we thus have 

0 = Im q(x) = (Imno) + (Imai)x + • • • + (Ima„_ 2 )x” _2 

for all x e R. This implies that Im r/o, . . . . Im a n -2 all equal 0 (by 4.7). Thus 
all the coefficients of q are real, as desired. Hence the desired factorization 
exists. 

Now we turn to the question of uniqueness of our factorization. A factor 
of p of the form x 2 + bjx + cj with bj 2 < 4 Cj can be uniquely written 
as (x — A 7 )(x — A j) with Ay e C. A moment’s thought shows that two 
different factorizations of p as an element of T^R) would lead to two different 
factorizations of p as an element of V(C), contradicting 4. 14. ■ 
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EXERCISES 4 


1 Verify all the assertions in 4.5 except the last one. 

2 Suppose m is a positive integer. Is the set 

{0} U {p e V(F) : deg p = m} 
a subspace of 'P(F)? 

3 Is the set 

{0} U {p e V(F) : deg p is even} 
a subspace of 'P(F)? 

4 Suppose m and n are positive integers with m < n, and suppose 
Ai, . . . , X m e F. Prove that there exists a polynomial p e V(F) with 
deg p — n such that 0 = p( Ai) = • • • = p{X m ) and such that p has no 
other zeros. 

5 Suppose m is a nonnegative integer, z \, . . . , z m +i are distinct elements 
of F, and wi , . . . , w m + \ e F. Prove that there exists a unique polynomial 
p e V m (F ) such that 

p(Zj) = Wj 

for j = 1 , . . . , m + 1 . 

[ This result can be proved without using linear algebra. However, try to 
find the clearer, shorter proof that uses some linear algebra .] 

6 Suppose p e V(C) has degree m. Prove that p has m distinct zeros if 
and only if p and its derivative p' have no zeros in common. 

7 Prove that every polynomial of odd degree with real coefficients has a 
real zero. 

8 Define T : P(R) -a- R r by 


Tp 


' P ~ P( 3) 
x — 3 

P\ 3) 


if jc 3, 
if x = 3. 


Show that Tp e 'P(R) for every polynomial p e 'P(R) and that 7 is a 
linear map. 
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9 Suppose p e V(C). Define q : C — »• C by 

q(z) = p(z)p(z). 

Prove that q is a polynomial with real coefficients. 

10 Suppose m is a nonnegative integer and p e V m (C) is such that there 
exist distinct real numbers xq,x\, . . . ,x m such that p(xj) e R for 
j = 0, 1 , ,m. Prove that all the coefficients of p arc real. 

11 Suppose p € V(¥) with p ^ 0. Let U = {pq : q € V(F)}. 

(a) Show that dim V(F)/U = deg p. 

(b) Find a basis of dim V(F )/U. 
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Statue of Italian mathematician 
Leonardo of Pisa ( 1170-1250 , 
approximate dates), also known as 
Fibonacci. Exercise 16 in Section 5.C 
shows how linear algebra can be used 
to find an explicit formula for the 
Fibonacci sequence. 


Eigenvalues, Eigenvectors, and 
Invariant Subspaces 


Linear maps from one vector space to another vector space were the objects 
of study in Chapter 3. Now we begin our investigation of linear maps from 
a finite-dimensional vector space to itself. Their study constitutes the most 
important part of linear algebra. 

Our standing assumptions are as follows: 

5.1 Notation F, V 

• F denotes R or C. 

• V denotes a vector space over F. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ invariant subspaces 

■ eigenvalues, eigenvectors, and eigenspaces 

■ each operator on a finite-dimensional complex vector space has an 
eigenvalue and an upper-triangular matrix with respect to some 
basis 
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CHAPTER 5 Eigenvalues, Eigenvectors, and Invariant Subspaces 


Invariant Subspaces 

In this chapter we develop the tools that will help us understand the structure 
of operators. Recall that an operator is a linear map from a vector space to 
itself. Recall also that we denote the set of operators on V by C(V)\ in other 
words, C(V ) = C{V. V). 

Let’s see how we might better understand what an operator looks like. 
Suppose T e C{V). If we have a direct sum decomposition 

V = Ui ©••• © U m , 

where each U j is a proper subspace of V, then to understand the behavior of 
T, we need only understand the behavior of each T \ u . ; here T \ u - denotes 
the restriction of T to the smaller domain Uj . Dealing with T \ jjj should be 
easier than dealing with T because U j is a smaller vector space than V. 

However, if we intend to apply tools useful in the study of operators (such 
as taking powers), then we have a problem: T\jjj may not map Uj into itself; 
in other words, T\jjj may not be an operator on Uj. Thus we are led to 
consider only decompositions of V of the form above where T maps each Uj 
into itself. 

The notion of a subspace that gets mapped into itself is sufficiently impor- 
tant to deserve a name. 

5.2 Definition invariant subspace 

Suppose T € C( V). A subspace U of V is called invariant under T if 
u € U implies Tu € U. 

In other words, U is invariant under T if T \ u is an operator on U. 

5.3 Example Suppose T e C(V). Show that each of the following 
subspaces of V is invariant under T : 

(a) {0}; 

(b) V ; 

(c) null T ; 

(d) range T. 


The most famous unsolved problem 
in functional analysis is called the 
invariant subspace problem. It 
deals with invariant subspaces of 
operators on infinite-dimensional 
vector spaces. 


5.A 
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Solution 

(a) If u E {0}, then u = 0 and hence T u = 0 e {0}. Thus {0} is invariant 
under T. 

(b) If u e V, then Tu € V. Thus V is invariant under T. 

(c) If u e null T, then Tu — 0, and hence T u e null T. Thus null T is 
invariant under T. 

(d) If u E range T, then Tu £ range T. Thus range T is invariant under T. 


Must an operator T E C(V) have any invariant subspaces other than {0} 
and V? Later we will see that this question has an affirmative answer if V is 
finite-dimensional and dim V > 1 (for F = C) or dim V > 2 (for F = R); 
see 5.21 and 9.8. 

Although null T and range T are invariant under T. they do not necessarily 
provide easy answers to the question about the existence of invariant subspaces 
other than {0} and V, because null T may equal {0} and range T may equal 
V (this happens when T is invertible). 

5.4 Example Suppose that T E jC(P(R)) is defined by Tp = p'. 
Then Vn (R), which is a subspace of T^R), is invariant under T because 
if p e?(R) has degree at most 4, then p' also has degree at most 4. 


Eigenvalues and Eigenvectors 

We will return later to a deeper study of invariant subspaces. Now we turn to an 
investigation of the simplest possible nontrivial invariant subspaces — invariant 
subspaces with dimension 1 . 

Take any v E V with v ^ 0 and let U equal the set of all scalar multiples 
of v: 

U — {Av : A e F} = span(v). 

Then U is a 1 -dimensional subspace of V (and every 1 -dimensional subspace 
of V is of this form for an appropriate choice of v). If U is invariant under an 
operator T E C(V), then Tv E U, and hence there is a scalar A E F such that 

Tv = Av. 

Conversely, if T v = Av for some A E F, then span(v) is a 1 -dimensional 
subspace of V invariant under T. 
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The equation 

T v = Av, 

which we have just seen is intimately connected with 1 -dimensional invariant 
subspaces, is important enough that the vectors v and scalars A satisfying it 
are given special names. 

5.5 Definition eigenvalue 

Suppose T e C(V). A number A e F is called an eigenvalue of T if 
there exists v € V such that v ^ 0 and Tv = Av. 


The word eigenvalue is half- 
German, half-English. The Ger- 
man adjective eigen means “own ” 
in the sense of characterizing an in- 
trinsic property. Some mathemati- 
cians use the term characteristic 
value instead of eigenvalue. 


The comments above show that T 
has a 1 -dimensional invariant subspace 
if and only if T has an eigenvalue. 

In the definition above, we require 
that v ^ 0 because every scalar A e F 
satisfies TO = AO. 


5.6 Equivalent conditions to be an eigenvalue 

Suppose V is finite-dimensional, T e C(V), and A e F. Then the 
following are equivalent: 

(a) A is an eigenvalue of T ; 

(b) T — XI is not injective; 

(c) T — XI is not surjective; 

(d) T — XI is not invertible. 

Proof Conditions (a) and (b) are equivalent because the equation T v = Av 
is equivalent to the equation (T — A/)v = 0. Conditions (b), (c), and (d) are 
equivalent by 3.69. ■ 

5.7 Definition eigenvector 

Suppose T € C{V) and A e F is an eigenvalue of T. A vector v € V is 
called an eigenvector of T corresponding to A if v ^ 0 and T v = Av. 


Recall that I e C( V) is the iden- 
tity operator defined by Iv = v for 
all v e V 
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Because Tv — Av if and only if (T — A/)v = 0, a vector v € V with v ^ 0 
is an eigenvector of T corresponding to A if and only if v e null(7 — XI). 

5.8 Example Suppose T e £( F 2 ) is defined by 

T(w,z) = ( z , w) . 

(a) Find the eigenvalues and eigenvectors of T if F = R. 

(b) Find the eigenvalues and eigenvectors of T if F = C. 

Solution 

(a) IfF = R, then T is a counterclockwise rotation by 90° about the 

origin in R 2 . An operator has an eigenvalue if and only if there exists a 

nonzero vector in its domain that gets sent by the operator to a scalar 

multiple of itself. A 90° counterclockwise rotation of a nonzero vector 
in R 2 obviously never equals a scalar multiple of itself. Conclusion: if 
F = R, then T has no eigenvalues (and thus has no eigenvectors). 

(b) To find eigenvalues of T, we must find the scalars A such that 

T(w, z ) = A (w, z ) 

has some solution other than w — z = 0. The equation above is 
equivalent to the simultaneous equations 

5.9 — z = Aw, w = Az. 

Substituting the value for w given by the second equation into the first 
equation gives 

— z = A 2 z. 

Now z cannot equal 0 [otherwise 5.9 implies that w = 0; we are 
looking for solutions to 5.9 where (w, z) is not the 0 vector], so the 
equation above leads to the equation 

— 1 = A 2 . 

The solutions to this equation are A = i and A = — i. You should 
be able to verify easily that i and —i are eigenvalues of T. Indeed, 
the eigenvectors corresponding to the eigenvalue i are the vectors of 
the form ( w, — wz ), with we C and w 7^ 0, and the eigenvectors 
corresponding to the eigenvalue — z are the vectors of the form (w, wz), 
with w e C and w/0. 
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Now we show that eigenvectors corresponding to distinct eigenvalues are 
linearly independent. 

5.10 Linearly independent eigenvectors 

Let T € C{V). Suppose . . . ,X m are distinct eigenvalues of T and 
vi , . . . , v m are corresponding eigenvectors. Then v\ , . . . , v m is linearly 
independent. 

Proof Suppose v \ , . . . , v m is linearly dependent. Let k be the smallest posi- 
tive integer such that 

5.11 vjfc € span(vi,...,vjfc_i); 

the existence of k with this property follows from the Linear Dependence 
Lemma (2.21). Thus there exist a\,. . . , <3^-1 e F such that 

5.12 Vfc = flivi + • • • + <Zfc-iVfc_i . 

Apply T to both sides of this equation, getting 

AjfcLfc ~ + • • • + iA£-iV£_i. 

Multiply both sides of 5.12 by A& and then subtract the equation above, getting 

0 = a i(Afc — Ai)vi + • • • + t(Ayt — A^_ 1 )v^_i. 

Because we chose k to be the smallest positive integer satisfying 5.11, 

vi \>k - 1 is linearly independent. Thus the equation above implies that all 

the a’s are 0 (recall that A^ is not equal to any of Ai A^_ 1 ). However, this 

means that \\ equals 0 (see 5.12), contradicting our hypothesis that \\ is an 
eigenvector. Therefore our assumption that vi , . . . , v m is linearly dependent 
was false. ■ 

The corollary below states that an operator cannot have more distinct 
eigenvalues than the dimension of the vector space on which it acts. 

5.13 Number of eigenvalues 

Suppose V is finite-dimensional. Then each operator on V has at most 
dim V distinct eigenvalues. 

Proof Let T e C{V). Suppose Ai, . . . , X m are distinct eigenvalues of T. 
Let vi, . . . , v m be corresponding eigenvectors. Then 5.10 implies that the list 
vi, . . . , v m is linearly independent. Thus m < dim V (see 2.23), as desired. ■ 
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Restriction and Quotient Operators 

If T e C(V) and U is a subspace of V invariant under T, then U determines 
two other operators T\u 6 C(U) and T/U € C(V/U) in a natural way, as 
defined below. 

5.14 Definition T\ v andT /U 

Suppose T e C{V) and U is a subspace of V invariant under T. 

• The restriction operator T\u e C{U ) is defined by 

T\u(u ) = Tu 

for u e U. 

• The quotient operator T / U e C(V / U) is defined by 

(T / U)(v + U) = Tv + U 

for v € V. 

For both the operators defined above, it is worthwhile to pay attention 
to their domains and to spend a moment thinking about why they are well 
defined as operators on their domains. First consider the restriction operator 
T\u e C(U), which is T with its domain restricted to U, thought of as 
mapping into U instead of into V. The condition that U is invariant under T 
is what allows us to think of T \ u as an operator on U, meaning a linear map 
into the same space as the domain, rather than as simply a linear map from 
one vector space to another vector space. 

To show that the definition above of the quotient operator makes sense, 
we need to verify that if v + U — w + U, then Tv + U = Tw + U. Flence 
suppose v + U — w + U. Thus v — w € U (see 3.85). Because U is invariant 
under T, we also have T(v — w) € U, which implies that Tv—Twe U, which 
implies that Tv + U = Tw + U, as desired. 

Suppose T is an operator on a finite-dimensional vector space V and U is 
a subspace of V invariant under T, with U ^ {0} and U ^ V. In some sense, 
we can learn about T by studying the operators T \ u and T/U, each of which 
is an operator on a vector space with smaller dimension than V. For example, 
proof 2 of 5.27 makes nice use of T / U. 

However, sometimes T \ u and T/U do not provide enough information 
about T. In the next example, both T\jj and T/U are 0 even though T is not 
the 0 operator. 
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5.15 Example Define an operator T e £(F 1 2 ) by T(x,y) = (y,0). Let 
U — {(x, 0) : x € F}. Show that 

(a) U is invariant under T and T\u is the 0 operator on U ; 

(b) there does not exist a subspace W of F 2 that is invariant under T and 
such that F 2 = U © W; 

(c) T/U is the 0 operator on F 2 / U. 

Solution 

(a) For (x,0) e U, we have T(x,0) = (0,0) e U. Thus U is invariant 
under T and T \ u is the 0 operator on U. 

(b) Suppose IF is a subspace of V such that F 2 = U © W. Because 
dim F 2 = 2 and dim U = 1 , we have dim W = 1 . If W were invariant 
under T, then each nonzero vector in W would be an eigenvector of T. 
However, it is easy to see that 0 is the only eigenvalue of T and that all 
eigenvectors of T are in U. Thus W is not invariant under T. 

(c) For (x,y) € F 2 , we have 

(T/U)((x,y) + U) = T(x,y) + U 
= (y,0) + U 
= 0 + U, 

where the last equality holds because (y, 0) e U. The equation above 
shows that T/U is the 0 operator. 


EXERCISES 5. A 

1 Suppose T € C(V) and U is a subspace of V. 

(a) Prove that if U C null T, then U is invariant under T. 

(b) Prove that if range T C U, then U is invariant under T. 

2 Suppose S,T € C(V) are such that ST — TS. Prove that null S is 
invariant under T. 
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3 Suppose S,T e C(V) are such that ST = TS. Prove that range S is 
invariant under T. 

4 Suppose that T € C(V) and U\, . . . , U m are subspaces of V invariant 
under T. Prove that U\ + • • • + U m is invariant under T. 

5 Suppose T € C(V). Prove that the intersection of every collection of 
subspaces of V invariant under T is invariant under T. 

6 Prove or give a counterexample: if V is finite-dimensional and U is a 
subspace of V that is invariant under every operator on V, then U = {0} 
or U = V. 

7 Suppose T € £(R 2 ) is defined by T(x,y) = (—3 y,x). Find the 
eigenvalues of T. 

8 Define T e £(F 2 ) by 

T{w,z ) = (z,w). 

Find all eigenvalues and eigenvectors of T. 

9 Define T e £(F 3 ) by 


T(zi,z 2 ,z 3 ) - (2z 2 , 0, 5z 3 ). 

Find all eigenvalues and eigenvectors of T. 

10 Define T e C(F n ) by 

T(x\, x 2 ,x 3 , . . . , x n ) = (xi,2x 2 ,3 x 3 ,...,nx„). 

(a) Find all eigenvalues and eigenvectors of T. 

(b) Find all invariant subspaces of T. 

11 Define T: V( R) — > ^(R) by Tp = p'. Find all eigenvalues and 
eigenvectors of T. 

12 Define T e C(V A ( R)) by 

(Tp)(x) = xp\x) 

for all xeR. Find all eigenvalues and eigenvectors of T. 

13 Suppose V is finite-dimensional, T e C(V), and A e F. Prove that there 
exists a € F such that \a — A| < and T — a I is invertible. 
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14 Suppose V — U © W, where U and W are nonzero subspaces of V. 
Define P g C{V) by P(u + w) = u for u g U and w € W. Find all 
eigenvalues and eigenvectors of P . 

15 Suppose T g C{V). Suppose S g C{V) is invertible. 

(a) Prove that T and S~ 1 TS have the same eigenvalues. 

(b) What is the relationship between the eigenvectors of T and the 
eigenvectors of S~ 1 TS1 

16 Suppose V is a complex vector space, T e C( V), and the matrix of T 
with respect to some basis of V contains only real entries. Show that if 
A is an eigenvalue of T, then so is A. 

17 Give an example of an operator T e £(R 4 ) such that T has no (real) 
eigenvalues. 

18 Show that the operator T e £(C°°) defined by 

T(zi,z 2 ,...) = (0,zi,z 2 ,...) 

has no eigenvalues. 

19 Suppose n is a positive integer and T e C( F") is defined by 

T(x - (xi H F x „ — ,xi H Fx„); 

in other words, T is the operator whose matrix (with respect to the 
standard basis) consists of all 1 ’s. Find all eigenvalues and eigenvectors 
of T. 

20 Find all eigenvalues and eigenvectors of the backward shift operator 
T g £(F°°) defined by 

r(zi,z 2 ,z 3 ,...) = (z 2 , z 3 ,...). 

21 Suppose T e C{V) is invertible. 

(a) Suppose A € F with A ^ 0. Prove that A is an eigenvalue of T if 
and only if j is an eigenvalue of T~ l . 

(b) Prove that T and T~ x have the same eigenvectors. 


SECTION 5. A Invariant Subspaces 


100 72 


22 


Suppose T e C{V) and there exist nonzero vectors v and w in V such 
that 

T v = 3 w and Tw = 3v. 

Prove that 3 or —3 is an eigenvalue of T. 


23 Suppose V is finite-dimensional and S,T e C{V). Prove that ST and 
T S have the same eigenvalues. 


24 Suppose A is an //-by-// matrix with entries in F. Define T e £(F n ) 
by Tx — Ax, where elements of F" are thought of as //-by- 1 column 
vectors. 


(a) Suppose the sum of the entries in each row of A equals 1 . Prove 
that 1 is an eigenvalue of T. 

(b) Suppose the sum of the entries in each column of A equals 1 . 
Prove that 1 is an eigenvalue of T. 


25 Suppose T e C(V) and u,v are eigenvectors of T such that u + v 
is also an eigenvector of T. Prove that u and v are eigenvectors of T 
corresponding to the same eigenvalue. 

26 Suppose T e £(F) is such that every nonzero vector in V is an eigen- 
vector of T. Prove that T is a scalar multiple of the identity operator. 

27 Suppose V is finite-dimensional and T e C{V) is such that every sub- 
space of V with dimension dim V — 1 is invariant under T. Prove that T 
is a scalar multiple of the identity operator. 

28 Suppose V is finite-dimensional with dim V > 3 and T e C{V) is such 
that every 2-dimensional subspace of V is invariant under T. Prove that 
T is a scalar multiple of the identity operator. 

29 Suppose T e C(V) and dimrange T — k. Prove that T has at most 
k + 1 distinct eigenvalues. 

30 Suppose T e £(R 3 ) and —4, 5, and 77 are eigenvalues of T. Prove that 
there exists x e R 3 such that T x — 9x — (—4, 5, 77). 

31 Suppose V is finite-dimensional and vi , . . . , v m is a list of vectors in V. 

Prove that v\ v m is linearly independent if and only if there exists 

T € C{V) such that vi, . . . , v m are eigenvectors of T corresponding to 
distinct eigenvalues. 
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32 Suppose Ai X n is a list of distinct real numbers. Prove that the list 

e^ lX , . . . , e^ nX is linearly independent in the vector space of real-valued 
functions on R. 

Hint: Let V = span(e^ lX , . . . , e^ nX ), and define an operator T e C( V) 
by Tf = f. Find eigenvalues and eigenvectors of T . 

33 Suppose T e C(V). Prove that T / (range T ) = 0. 

34 Suppose T e C( V). Prove that T /(null T) is injective if and only if 
(null T) n (range T) = {0}. 

35 Suppose V is finite-dimensional, T e C(V), and U is invariant under T. 
Prove that each eigenvalue of T / U is an eigenvalue of T. 

[The exercise below asks you to verify that the hypothesis that V is 
finite-dimensional is needed for the exercise above.] 

36 Give an example of a vector space V, an operator T e C(V), and 
a subspace U of V that is invariant under T such that T/U has an 
eigenvalue that is not an eigenvalue of T. 
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Eigenvectors and Upper-Triangular 
Matrices 


Polynomials Applied to Operators 

The main reason that a richer theory exists for operators (which map a vector 
space into itself) than for more general linear maps is that operators can be 
raised to powers. We begin this section by defining that notion and the key 
concept of applying a polynomial to an operator. 

If T e C{V), then T T makes sense and is also in C{V). We usually write 
T 2 instead of T T. More generally, we have the following definition. 


5.16 Definition T m 

Suppose T € C(V) and m is a positive integer. 
• T m is defined by 

J^m y J 


• T° is defined to be the identity operator I on V. 

• If T is invertible with inverse T~ l , then T~ m is defined by 

T ~ m _ (7-1)™. 

You should verify that if T is an operator, then 

pmpn rpm+n ^rpm\n pmn 

where m and n are allowed to be arbitrary integers if T is invertible and 
nonnegative integers if T is not invertible. 


5.17 Definition p(T ) 

Suppose T e C{V) and p e V(F) is a polynomial given by 

p(z) = a 0 + aiz + a 2 z 2 + h a m z m 

for z e F. Then p( T) is the operator defined by 

p(T) = ciqI + a i T + a 2 T ~ + • • • + a m T m . 

This is a new use of the symbol p because we are applying it to operators, 
not just elements of F. 
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5.18 Example Suppose D e £('P(R)) is the differentiation operator 
defined by Dq = q' and p is the polynomial defined by p(x ) = 7 — 3x + 5x 2 . 
Then p(D) = 11 — 3D + 5 Z) 2 ; thus 

( p(D))q = lq — 3 q' + 5 q" 

for every q e 'P(R). 


If we fix an operator T e £{ V), then the function from T^F) to £( V) 
given by p t-> p(T) is linear, as you should verify. 


5.19 Definition product of polynomials 
If /?, q e 'P(F), then pq € V(F) is the polynomial defined by 

ipq)(z) = p(z)q(z) 

for z e F. 

Any two polynomials of an operator commute, as shown below. 

5.20 Multiplicative properties 

Suppose p,q € V(F) and T e C(V). 

Then 

(a) {pq){T) = p(T)q(T); 

(b) p(T)q(T) = q(T)p(T). 

Proof 

(a) Suppose p(z) = Yl'j=o a j z '’ and q(z) — Ylk=o^k zk for z € F. 
Then 

m n 

(pq)(z) = EE a jb k z ]+k . 

j=0k=0 

Thus 

m n 

(. pq)(T ) = EE ajb k Tj +k 

j=0k=0 

m n 

= (E^ 7 ' y )(E 4 ‘ 7 ' t ) 

7=0 k = 0 

= p(T)q(T). 

(b) Part (a) implies p(T)q(T) = (pq)(T) = ( qp)(T ) = q(T)p(T). m 


Part (a) holds because when ex- 
panding a product of polynomials 
using the distributive property, it 
does not matter whether the sym- 
bol is z or T. 
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Existence of Eigenvalues 

Now we come to one of the central results about operators on complex vector 
spaces. 

5.21 Operators on complex vector spaces have an eigenvalue 

Every operator on a finite-dimensional, nonzero, complex vector space 
has an eigenvalue. 

Proof Suppose V is a complex vector space with dimension n > 0 and 
T € C(V). Choose v € V with v ^ 0. Then 

v, Tv, T 2 v, ...,T n v 

is not linearly independent, because V has dimension n and we have n + 1 
vectors. Thus there exist complex numbers ao, ... ,a n , not all 0, such that 

0 = aov + a\Tv 4 + a n T n v. 

Note that a\, ... ,a n cannot all be 0, because otherwise the equation above 
would become 0 = ciqv, which would force cio also to be 0. 

Make the a’ s the coefficients of a polynomial, which by the Fundamental 
Theorem of Algebra (4.14) has a factorization 

«o + a\z H b a n z n = c(z - AO ■ ■ ■ (z - X m ), 


where c is a nonzero complex number, each Xj is in C, and the equation holds 
for all z e C (here m is not necessarily equal to n, because a n may equal 0). 
We then have 


0 = Oqv + a\Tv 4 1- a n T n v 

= ( aol + a\ T + • • • + a n T n )v 
= c(T-X 1 I)-(T-X m I)v. 


Thus T — Xj I is not injective for at least one j . In other words, T has an 
eigenvalue. ■ 

The proof above depends on the Fundamental Theorem of Algebra, which 
is typical of proofs of this result. See Exercises 16 and 17 for possible ways to 
rewrite the proof above using the idea of the proof in a slightly different form. 
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Upper- Triangular Matrices 

In Chapter 3 we discussed the matrix of a linear map from one vector space 
to another vector space. That matrix depended on a choice of a basis of each 
of the two vector spaces. Now that we are studying operators, which map a 
vector space to itself, the emphasis is on using only one basis. 


5.22 Definition matrix of an operator, M.(T) 

Suppose T e C(V) and vi, . . . , v n is a basis of V. The matrix of T with 
respect to this basis is the n-by-n matrix 

^ ^4t,i . . . Ai, n ^ 


M(T) = 


\ Afi'i . . . A n , n J 

whose entries Aj^ are defined by 


Tv t t — Ai^vi + • ■ ■ + A n jzV n . 


If the basis is not clear from the context, then the notation 
M(T, (vi, . . . , v„)) is used. 


Note that the matrices of operators are square arrays, rather than the more 
general rectangular arrays that we considered earlier for linear maps. 

If T is an operator on F” and no 
basis is specified, assume that the basis 
in question is the standard one (where 
the j th basis vector is 1 in the j th slot 
and 0 in all the other slots). You can 
then think of the j th column of M ( T ) as T applied to the j th basis vector. 

5.23 Example DefineT e £(F 3 ) by T(x, y, z) = (2x+y, 5y + 3z, 8z). 
Then 

/ 2 1 0 
M(T) =053 

V 0 0 8 

A central goal of linear algebra is to show that given an operator T e C(V), 
there exists a basis of V with respect to which T has a reasonably simple 
matrix. To make this vague formulation a bit more precise, we might try to 
choose a basis of V such that Ai (T) has many 0’s. 



The k th column of the matrix 
A4(T) is formed from the coeffi- 
cients used to write T v k as a linear 
combination of v i, . . . , v„. 
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If V is a finite-dimensional complex vector space, then we already know 
enough to show that there is a basis of V with respect to which the matrix of 
T has 0’s everywhere in the first column, except possibly the first entry. In 
other words, there is a basis of V with respect to which the matrix of T looks 
like 

( X \ 

0 * 

V 0 

here the * denotes the entries in all the columns other than the first column. 
To prove this, let X be an eigenvalue of T (one exists by 5.21) and let v be a 
corresponding eigenvector. Extend v to a basis of V. Then the matrix of T 
with respect to this basis has the form above. 

Soon we will see that we can choose a basis of V with respect to which 
the matrix of T has even more 0’s. 

5.24 Definition diagonal of a matrix 

The diagonal of a square matrix consists of the entries along the line from 
the upper left corner to the bottom right corner. 

For example, the diagonal of the matrix in 5.23 consists of the entries 
2,5,8. 

5.25 Definition upper -triangular matrix 

A matrix is called upper triangular if all the entries below the diagonal 
equal 0. 

For example, the matrix in 5.23 is upper triangular. 

Typically we represent an upper-triangular matrix in the form 

* Xi * ^ 

\ o X n J 

the 0 in the matrix above indicates 
that all entries below the diagonal in 
this n-hy-n matrix equal 0. Upper- 
triangular matrices can be considered 
reasonably simple — for n large, almost 
half its entries in an n-hy-n upper- 
triangular matrix are 0. 


We often use * to denote matrix en- 
tries that we do not know about or 
that are irrelevant to the questions 
being discussed. 
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The following proposition demonstrates a useful connection between 
upper-triangular matrices and invariant subspaces. 

5.26 Conditions for upper-triangular matrix 

Suppose T € C(V) and vi , . . . , v n is a basis of V. Then the following are 
equivalent: 

(a) the matrix of T with respect to vi, . . . , v n is upper triangular; 

(b) T Vj e span(vi , ... ,Vj) for each j = 1 

(c) span(vi Vj) is invariant under T for each j = 1 

Proof The equivalence of (a) and (b) follows easily from the definitions and 
a moment’s thought. Obviously (c) implies (b). Hence to complete the proof, 
we need only prove that (b) implies (c). 

Thus suppose (b) holds. Fix j e {1 , n). From (b), we know that 

Tv i e span(vi) C span(vi, . . . , v 7 ); 

T v 2 6 spanfvi , v 2 ) C spanfvi , . . . , Vj ); 


T Vj e spanfvi, . . . , Vj). 

Thus if v is a linear combination of vi , ,Vj, then 

T v e span(vi, . . . , Vj). 

In other words, span(vi, . . . , Vj) is invariant under T, completing the proof. ■ 

Now we can prove that for each 
operator on a finite-dimensional com- 
plex vector space, there is a basis of the 
vector space with respect to which the 
matrix of the operator has only 0’s be- 
low the diagonal. In Chapter 8 we will 
improve even this result. 

Sometimes more insight comes from 
seeing more than one proof of a theo- 
rem. Thus two proofs are presented of 
the next result. Use whichever appeals 
more to you. 


The next result does not hold on 
real vector spaces, because the first 
vector in a basis with respect to 
which an operator has an upper- 
triangular matrix is an eigenvector 
of the operator. Thus if an opera- 
tor on a real vector space has no 
eigenvalues [see 5.8(a) for an ex- 
ample], then there is no basis with 
respect to which the operator has 
an upper-triangular matrix. 
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5.27 Over C, every operator has an upper-triangular matrix 

Suppose V is a finite-dimensional complex vector space and T e C(V). 
Then T has an upper-triangular matrix with respect to some basis of V. 

Proof 1 We will use induction on the dimension of V. Clearly the desired 
result holds if dim V = 1 . 

Suppose now that dim V > 1 and the desired result holds for all complex 
vector spaces whose dimension is less than the dimension of V. Let A be any 
eigenvalue of T (5.21 guarantees that T has an eigenvalue). Let 

U = range (T — XI). 

Because T — XI is not surjective (see 3.69), dim U < dim V. Furthermore, 
U is invariant under T. To prove this, suppose u e U. Then 

T u — (T — XI )u + Xu. 

Obviously (T — XI )u € U (because U equals the range of T — XI) and 
Xu € U. Thus the equation above shows that Tu e U. Flence U is invariant 
under T. as claimed. 

Thus T\u is an operator on U. By our induction hypothesis, there is a 
basis ui, ... ,u m of U with respect to which T\u has an upper-triangular 
matrix. Thus for each j we have (using 5.26) 

5.28 Tuj — ( T\u){iij ) e span(wi, . . . , Uj). 

Extend u\ , . . . , u m to a basis u i u m ,v\, . . . ,v n of V. For each k, we 

have 

Tv k = (T — XI)v k + Xv k . 

The definition of U shows that (T — X I)v k € U = span(ui, . . . , u m ). Thus 
the equation above shows that 

5.29 Tv k e span(wi , . . . ,u m ,v\, . . . , v k ). 

From 5.28 and 5.29, we conclude (using 5.26) that T has an upper- 
triangular matrix with respect to the basis u\, . . . ,u m ,v\, . . . ,v n of V, as 
desired. ■ 
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Proof 2 We will use induction on the dimension of V. Clearly the desired 
result holds if dim V = 1 . 

Suppose now that dim V — n > 1 and the desired result holds for all 
complex vector spaces whose dimension is n — 1. Let vi be any eigenvector 
of T (5.21 guarantees that T has an eigenvector). Let U — span(vi). Then U 
is an invariant subspace of T and dim (7=1. 

Because dim V/U = n — 1 (see 3.89), we can apply our induction hy- 
pothesis to T/U € C{V/U). Thus there is a basis V2 + U v n + U of 

V/U such that T/U has an upper-triangular matrix with respect to this basis. 
Hence by 5.26, 


( T/U)( Vj +U)e span(v 2 + U,...,vj + U) 

for each j — 2 ,...,«. Unraveling the meaning of the inclusion above, we 
see that 

Tvj e span(vi, . . . ,v/) 

for each j — 1 Thus by 5.26, T has an upper-triangular matrix 
with respect to the basis vi , . . . , v n of V, as desired (it is easy to verify that 
vi , . . . , v n is a basis of V ; see Exercise 13 in Section 3.E for a more general 
result). ■ 

How does one determine from looking at the matrix of an operator whether 
the operator is invertible? If we are fortunate enough to have a basis with 
respect to which the matrix of the operator is upper triangular, then this 
problem becomes easy, as the following proposition shows. 


5.30 Determination of invertibility from upper-triangular matrix 

Suppose T e C(V) has an upper-triangular matrix with respect to some 
basis of V. Then T is invertible if and only if all the entries on the diagonal 
of that upper-triangular matrix are nonzero. 


Proof Suppose vi, . . . , v n is a basis of V with respect to which T has an 
upper-triangular matrix 


5.31 


M(T) = 


/ Ai 

^2 

* ) 

V o 

^■n ) 


We need to prove that T is invertible if and only if all the A j ’s are nonzero. 
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First suppose the diagonal entries ... ,X n are all nonzero. The upper- 
triangular matrix in 5.31 implies that Tv i = Aivi. Because Ai 7^ 0, we have 
r(vi/Ai) = vi; thus vi e range T. 

Now 

T(v 2/X2) — av 1 + v 2 

for some a e F. The left side of the equation above and av\ are both in 
range T ; thus v 2 e range T. 

Similarly, we see that 

T(v 3/A3) = &Vl + CV2 + V’3 

for some 6, c e F. The left side of the equation above and bv \ , cv 2 are all in 
range T ; thus V3 e range 7. 

Continuing in this fashion, we conclude that vi v n 6 range T. Be- 

cause vi , . . . , v n is a basis of V, this implies that range T — V. In other words, 
T is surjective, ffence T is invertible (by 3.69), as desired. 

To prove the other direction, now suppose that T is invertible. This implies 
that Ai 7^ 0, because otherwise we would have Tv\ =0. 

Let 1 < j < n, and suppose A j — 0. Then 5.31 implies that T maps 
span(vi, . . . , Vj) into span(vi, . . . , v/_ 1). Because 


dimspan(i’i, ...,Vj) = j and dimspan(vi, . . . , v/_i) = j — 1, 

this implies that T restricted to dim span(vi v/) is not injective (by 3.23). 

Thus there exists v e span(vi , ... ,Vj) such that v 7^ 0 and T v = 0. Thus T 
is not injective, which contradicts our hypothesis (for this direction) that T is 
invertible. This contradiction means that our assumption that Xj — 0 must be 
false. Hence Xj 7^ 0, as desired. ■ 

As an example of the result above, we see that the operator in Example 5.23 
is invertible. 

Unfortunately no method exists for 
exactly computing the eigenvalues of 
an operator from its matrix. However, 
if we are fortunate enough to find a ba- 
sis with respect to which the matrix of 
the operator is upper triangular, then the 
problem of computing the eigenvalues 
becomes trivial, as the following propo- 
sition shows. 


Powerful numeric techniques exist 
for finding good approximations to 
the eigenvalues of an operator from 
its matrix. 
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5.32 Determination of eigenvalues from upper-triangular matrix 

Suppose T e C{V) has an upper-triangular matrix with respect to some 
basis of V. Then the eigenvalues of T are precisely the entries on the 
diagonal of that upper-triangular matrix. 

Proof Suppose vi, . . . , v n is a basis of V with respect to which T has an 
upper-triangular matrix 


/ Ai 

M(T) = 


A2 


V 0 


Let AeF. Then 


* > 


M(T — A/) 


/ Ai -A 

A 2 — A 


V 0 


* 


A n — A 


\ 


Hence T — XI is not invertible if and only if A equals one of the numbers 
Ai, . . . , A„ (by 5.30). Thus A is an eigenvalue of T if and only if A equals one 
of the numbers A 1 , . . . , X n . ■ 


5.33 Example DefineT € £(F 3 )by T(x,y,z) — (2x + y,5y + 3z,Sz). 
What are the eigenvalues of T? 

Solution The matrix of T with respect to the standard basis is 

/ 2 1 0 
M(T) =053 

V 0 0 8 

Thus M. (T) is an upper-triangular matrix. Now 5.32 implies that the eigen- 
values of T are 2, 5, and 8. 



Once the eigenvalues of an operator on F” are known, the eigenvectors 
can be found easily using Gaussian elimination. 
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EXERCISES 5.B 


1 Suppose 7 € 7(E) and there exists a positive integer n such that T n = 0. 

(a) Prove that I — 7 is invertible and that 

(7 - 7T 1 = 7 + 7 + ••• + 7 n_1 . 

(b) Explain how you would guess the formula above. 

2 Suppose 7 G 7(E) and (7 - 27) (7 - 37)(7 - 47) = 0. Suppose A is 
an eigenvalue of 7. Prove that A = 2 or A = 3 or A = 4. 

3 Suppose 7 G 7(E) and 7 2 = 7 and —1 is not an eigenvalue of 7. Prove 
that 7 = 7. 

4 Suppose 7 G 7(E) and P 2 = P . Prove that E = null P © range P . 

5 Suppose 5,7 G 7(E) and S is invertible. Suppose p G 7(F) is a 
polynomial. Prove that 

p(STS _1 ) = Sp(7)S -1 . 

6 Suppose 7 G 7(E) and U is a subspace of E invariant under 7. Prove 
that U is invariant under p(T) for every polynomial p e 7(F). 

7 Suppose 7 G 7(E). Prove that 9 is an eigenvalue of 7 2 if and only if 3 
or —3 is an eigenvalue of 7. 

8 Give an example of 7 e 7(R 2 ) such that 7 4 = — 1. 

9 Suppose E is finite-dimensional, 7 € 7(E), and v € E with v 7 ^ 0. 
Let p be a nonzero polynomial of smallest degree such that p(T)v = 0. 
Prove that every zero of p is an eigenvalue of 7. 

10 Suppose 7 € 7(E) and v is an eigenvector of 7 with eigenvalue A. 
Suppose p G 7(F). Prove that p{T)v — p( A)v. 

11 Suppose F = C, 7 g 7(E), p e 7(C) is a polynomial, and a G C. 
Prove that a is an eigenvalue of p{T) if and only if a — 77 (A) for some 
eigenvalue A of 7. 

12 Show that the result in the previous exercise does not hold if C is replaced 
with R. 
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13 Suppose W is a complex vector space and T e C{ W) has no eigenvalues. 
Prove that every subspace of W invariant under T is either {0} or infinite- 
dimensional. 

14 Give an example of an operator whose matrix with respect to some basis 
contains only 0’s on the diagonal, but the operator is invertible. 

[The exercise above and the exercise below show that 5.30 fails without 
the hypothesis that an upper-triangular matrix is under consideration. ] 

15 Give an example of an operator whose matrix with respect to some basis 
contains only nonzero numbers on the diagonal, but the operator is not 
invertible. 

16 Rewrite the proof of 5.21 using the linear map that sends p e V n (C) to 
(p(T))v € V (and use 3.23). 

17 Rewrite the proof of 5.21 using the linear map that sends p e V n 2 (C) to 
p{T) € C(V) (and use 3.23). 

18 Suppose V is a finite-dimensional complex vector space and T g C{ V). 
Define a function / : C — »■ R by 

/(A) = dimrange(r — XI). 

Prove that / is not a continuous function. 

19 Suppose V is finite-dimensional with dim V > 1 and T g C(V). Prove 
that 

{p(T) : p g V(F)} + C{V). 

20 Suppose V is a finite-dimensional complex vector space and T e C(V). 
Prove that T has an invariant subspace of dimension k for each k — 
1 , . . . . dim V. 
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Eigenspaces and Diagonal Matrices 

5.34 Definition diagonal matrix 

A diagonal matrix is a square matrix that is 0 everywhere except possibly 
along the diagonal. 


5.C 


5.35 Example 

( § 

0 

°\ 


° 

5 

° 


V 0 

0 

5 ) 

is a diagonal matrix. 





Obviously every diagonal matrix is upper triangular. In general, a diagonal 
matrix has many more 0’s than an upper-triangular matrix. 

If an operator has a diagonal matrix with respect to some basis, then the 
entries along the diagonal are precisely the eigenvalues of the operator; this 
follows from 5.32 (or find an easier proof for diagonal matrices). 

5.36 Definition eigenspace, E(X,T) 

Suppose T e C.(V) and A e F. The eigenspace of T corresponding to A, 
denoted E(X, T), is defined by 

E(X,T) = null(r-A/). 

In other words, E(X,T) is the set of all eigenvectors of T corresponding 
to A, along with the 0 vector. 

For T e C(V) and A e F, the eigenspace E(X,T) is a subspace of V 
(because the null space of each linear map on V is a subspace of V ). The 
definitions imply that A is an eigenvalue of T if and only if E(X, T) ^ {0}. 


5.37 Example Suppose the matrix of an operator T e C( V) with respect 
to a basis vq , V 2 , V 3 of V is the matrix in Example 5.35 above. Then 

E( 8 , T) - span(vi), E(5, T ) = span(v 2 , v 3 ). 


If A is an eigenvalue of an operator T e C{ V), then T restricted to 
E{X , T) is just the operator of multiplication by A. 
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5.38 Sum of eigenspaces is a direct sum 

Suppose V is finite-dimensional and T € C(V). Suppose also that 
A i X m are distinct eigenvalues of T. Then 

E(X u T) + --- + E(X m ,T) 


is a direct sum. Furthermore, 

dim E(X\, T) + • • • + dim E(X m , T) < dim V. 

Proof To show that E( Ai, T) + • ■ ■ + E( X m , T) is a direct sum, suppose 

wi + ■ ■ • + u m = 0, 

where each Uj is in E(X, T). Because eigenvectors corresponding to distinct 
eigenvalues are linearly independent (see 5.10), this implies that each Uj 

equals 0. This implies (using 1.44) that E( Ai , T) 4 \- E(X m , T) is a direct 

sum, as desired. 

Now 

dimitfAi, T) -\ b dim£'(A m , T) — dim(£’(Ai, T) © ••• © E( X m , T )) 

< dim V, 

where the equality above follows from Exercise 16 in Section 2.C. ■ 

5.39 Definition diagonalizable 

An operator T e C(V) is called diagonalizable if the operator has a 
diagonal matrix with respect to some basis of V. 


5.40 Example Define T e £(R 2 ) by 

TO c, y) = (4Lv + ly, -20a + 74y). 
The matrix of T with respect to the standard basis of R 2 is 



which is not a diagonal matrix. However, T is diagonalizable, because the 
matrix of T with respect to the basis (1, 4), (7, 5) is 



as you should verify. 


SECTION 5.C Eigenspaces and Diagonal Matrices 


157 


5.41 Conditions equivalent to diagonalizability 

Suppose V is finite-dimensional and T e C{V). Let Ai, . . . , A m denote 
the distinct eigenvalues of T. Then the following are equivalent: 

(a) T is diagonalizable; 

(b) V has a basis consisting of eigenvectors of T ; 

(c) there exist 1 -dimensional subspaces U\ U n of V, each invariant 

under T, such that 

V = £/i © ••• © U n \ 

(d) V = E(X u T)e---e E(X m ,T); 

(e) dim V = dim ft (A i, T) + ■ ■ ■ + dim ft(A m , T). 

Proof An operator T € C( V ) has a diagonal matrix 



with respect to a basis vi , . . . , v n of V if and only if T vj = A jvj for each j . 
Thus (a) and (b) are equivalent. 

Suppose (b) holds; thus V has a basis v\, ... ,v n consisting of eigenvectors 
of T. For each j , let Uj — span(v 7 ). Obviously each Uj is a 1 -dimensional 

subspace of V that is invariant under T. Because iq v n is a basis of V, 

each vector in V can be written uniquely as a linear combination of vi , . . . , v n . 

In other words, each vector in V can be written uniquely as a sum u \ H \-u n , 

where each u j is in Uj. Thus V = U\ © • • • © U n . Hence (b) implies (c). 

Suppose now that (c) holds; thus there are 1 -dimensional subspaces 
U\, ... ,U n of V, each invariant under T, such that V = Ui © ■ • • © U n . 
For each j , let vj be a nonzero vector in Uj . Then each Vj is an eigenvector 

of T. Because each vector in V can be written uniquely as a sum u i H \ -u n , 

where each Uj is in Uj (so each Uj is a scalar multiple of Vj), we see that 
vi , . . . , v„ is a basis of V. Thus (c) implies (b). 

At this stage of the proof we know that (a), (b), and (c) are all equivalent. 
We will finish the proof by showing that (b) implies (d), that (d) implies (e), 
and that (e) implies (b). 
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Suppose (b) holds; thus V has a basis consisting of eigenvectors of T. 
Hence every vector in V is a linear combination of eigenvectors of T, which 
implies that 

V = E(X lt T) + ... + E(X m , T). 

Now 5.38 shows that (d) holds. 

That (d) implies (e) follows immediately from Exercise 16 in Section 2.C. 
Finally, suppose (e) holds; thus 

5.42 dim V = dim E(X \ , T) + ■ • • + dim E(X m , T). 

Choose a basis of each E{Xj , T)\ put all these bases together to form a list 
vi , . . . , v n of eigenvectors of T, where n = dim V (by 5.42). To show that 
this list is linearly independent, suppose 


U\V\ + • • • + a n Vn — 0, 

where a\ a n e F. For each j — 1 m, let Uj denote the sum of all 

the terms such that \\ e E(Xj , T ). Thus each Uj is in E(Xj , T), and 

Mi + • • • + u m = 0. 

Because eigenvectors corresponding to distinct eigenvalues are linearly inde- 
pendent (see 5.10), this implies that each Uj equals 0. Because each Uj is a 
sum of terms a^v^, where the v^’s were chosen to be a basis of E(Xj, T ). this 
implies that all the equal 0. Thus vi, . . . , v n is linearly independent and 
hence is a basis of V (by 2.39). Thus (e) implies (b), completing the proof. ■ 

Unfortunately not every operator is diagonalizable. This sad state of affairs 
can arise even on complex vector spaces, as shown by the next example. 

5.43 Example Show that the operator T e £(C 2 ) defined by 

T(w,z) - (z, 0) 


is not diagonalizable. 

Solution As you should verify, 0 is the only eigenvalue of T and furthermore 
E(0,T) = {(w, 0) € C 2 :w € C}. 

Thus conditions (b), (c), (d), and (e) of 5.41 are easily seen to fail (of 
course, because these conditions are equivalent, it is only necessary to check 
that one of them fails). Thus condition (a) of 5.41 also fails, and hence T is 
not diagonalizable. 
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The next result shows that if an operator has as many distinct eigenvalues 
as the dimension of its domain, then the operator is diagonalizable. 

5.44 Enough eigenvalues implies diagonalizability 

If T € C( V) has dim V distinct eigenvalues, then T is diagonalizable. 

Proof Suppose T e C{V) has dim V distinct eigenvalues Ai, . . . , A^m v- 
For each j , let vj € V be an eigenvector corresponding to the eigenvalue A j . 
Because eigenvectors corresponding to distinct eigenvalues are linearly inde- 
pendent (see 5. 10 ), vi , . . . , v^im v i s linearly independent. A linearly indepen- 
dent list of dim V vectors in V is a basis of V (see 2.39); thus vi , . . . , v nim v 
is a basis of V. With respect to this basis consisting of eigenvectors, T has a 
diagonal matrix. ■ 


5.45 Example Define T € £(F 3 ) by T(x, y, z) = (2x + y, 5 y + 3z, 8 z). 
Find a basis of F 3 with respect to which T has a diagonal matrix. 

Solution With respect to the standard basis, the matrix of T is 



The matrix above is upper triangular. Thus by 5.32, the eigenvalues of T are 
2, 5, and 8. Because T is an operator on a vector space with dimension 3 and 
T has three distinct eigenvalues, 5.44 assures us that there exists a basis of F 3 
with respect to which T has a diagonal matrix. 

To find this basis, we only have to find an eigenvector for each eigenvalue. 
In other words, we have to find a nonzero solution to the equation 


T(x, y,z) = A (x,y,z) 


for A = 2, then for A = 5, and then for A = 8. These simple equations are 
easy to solve: for A = 2 we have the eigenvector (1,0, 0); for A = 5 we have 
the eigenvector (1, 3, 0); for A = 8 we have the eigenvector (1, 6, 6). 

Thus (1, 0, 0), (1, 3, 0), (1, 6, 6) is a basis of F 3 , and with respect to this 
basis the matrix of T is 
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The converse of 5.44 is not true. For example, the operator T defined on 
the three-dimensional space F 1 2 3 4 by 

r(zi,z 2 ,z 3 ) = (4 zi,4z 2 ,5z 3 ) 

has only two eigenvalues (4 and 5), but this operator has a diagonal matrix 
with respect to the standard basis. 

In later chapters we will find additional conditions that imply that certain 
operators are diagonalizable. 

EXERCISES 5.C 

1 Suppose T e C{V) is diagonalizable. Prove that V = null T © range T. 

2 Prove the converse of the statement in the exercise above or give a 
counterexample to the converse. 

3 Suppose V is finite-dimensional and T e C(V). Prove that the following 
are equivalent: 

(a) V = null T © range T. 

(b) V = null T + range T. 

(c) null T n range T = {0}. 

4 Give an example to show that the exercise above is false without the 
hypothesis that V is finite-dimensional. 

5 Suppose V is a finite-dimensional complex vector space and T € C(V). 
Prove that T is diagonalizable if and only if 

V = nuU(T - XI) © range (T - XI) 

for every A e C. 

6 Suppose V is finite-dimensional, T e C( V) has dim V distinct eigenval- 
ues, and S € C{ V) has the same eigenvectors as T (not necessarily with 
the same eigenvalues). Prove that ST = TS. 

1 Suppose T e C(V) has a diagonal matrix A with respect to some basis 
of V and that A e F. Prove that A appears on the diagonal of A precisely 
dim is (A, T) times. 

8 Suppose T e £(F 5 6 * 8 ) and dim £'(8, T) — 4. Prove that T — 21 or T — 61 
is invertible. 
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9 Suppose T e C(V) is invertible. Prove that E(X, T) — F(j- T 1 ) for 
every A e F with A ^ 0. 

10 Suppose that V is finite-dimensional and T e C{V). Let Ai X m 

denote the distinct nonzero eigenvalues of T. Prove that 

dim E(X\, T) + ••• + dim E(X m , T) < dim range T. 

11 Verify the assertion in Example 5.40. 

12 Suppose R.T € £(F 3 ) each have 2, 6, 7 as eigenvalues. Prove that there 
exists an invertible operator S € £(F 3 ) such that R — S ~ 1 TS . 

13 Find R,T e £(F 4 ) such that R and T each have 2, 6, 7 as eigenvalues, 
R and T have no other eigenvalues, and there does not exist an invertible 
operator S e £(F 4 ) such that R — S~ 1 TS. 

14 Find T e £(C 3 ) such that 6 and 7 are eigenvalues of T and such that T 
does not have a diagonal matrix with respect to any basis of C 3 . 

15 Suppose T € C( C 3 ) is such that 6 and 7 are eigenvalues of T. Fur- 
thermore, suppose T does not have a diagonal matrix with respect 
to any basis of C 3 . Prove that there exists (x, y, z) e F 3 such that 
T(x, y, z) = (17 + 8x, V5 + 8_v, 2 n + 8 z). 

16 The Fibonacci sequence F\ , F 2 , . . . is defined by 

F\ = 1, F 2 = 1, and F n = F n - 2 + F n ~ 1 for n > 3. 

Define T € F(R 2 ) by T(x, y ) = (y, x + y). 

(a) Show that T n (0, 1) = (F n , F n + 1 ) for each positive integer n. 

(b) Find the eigenvalues of T. 

(c) Find a basis of R 2 consisting of eigenvectors of T. 

(d) Use the solution to part (c) to compute T n ( 0, 1). Conclude that 



for each positive integer n . 

(e) Use part (d) to conclude that for each positive integer n, the 
Fibonacci number F n is the integer that is closest to 


1 /1 + V5\" 
V5V 2 ) 



CHAPTER 




Woman teaching 
geometry, from a 
fourteenth-century 
edition of Euclid’s 
geometry book. 


Inner Product Spaces 


In making the definition of a vector space, we generalized the linear structure 
(addition and scalar multiplication) of R 2 and R 3 . We ignored other important 
features, such as the notions of length and angle. These ideas are embedded 
in the concept we now investigate, inner products. 

Our standing assumptions are as follows: 

6.1 Notation F, V 

• F denotes R or C. 

• V denotes a vector space over F. 
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CHAPTER 6 Inner Product Spaces 


6.A 


Inner Products and Norms 


Inner Products 

To motivate the concept of inner prod- 
uct, think of vectors in R 2 and R 3 as 
arrows with initial point at the origin. 
The length of a vector a in R 2 or R 3 
is called the norm of x, denoted ||a||. 
Thus for x = (jci, X2) 6 R 2 , we have 

Ml = v*i 2 + x 2 2 . 

Similarly, i f a = (ai, X2, A3) 6 R 3 , 
then || a || = d x i 2 + x 2 2 + *3 2 - 
Even though we cannot draw pictures in higher dimensions, the gener- 
alization to R" is obvious: we define the norm of a = (x\ x n ) e R" 

by 

II A II = s / x \ 2 + i- x n 2 - 

The norm is not linear on R". To inject linearity into the discussion, we 
introduce the dot product. 

6.2 Definition dot product 

For a, y € R", the dot product of a and y, denoted a ■ y, is defined by 

x ■ y — A1V1 H b a n y n , 

where a = (ai , . . . , x») and y = (yi y n ). 

Note that the dot product of two vec- 
tors in R" is a number, not a vector. Ob- 
viously a ■ a = || a || 2 for all a e R”. 
The dot product on R” has the follow- 
ing properties: 

• a ■ a > 0 for all a e R" ; 

• a ■ a = 0 if and only if a = 0; 

• for y e R" fixed, the map from R" to R that sends a € R” to a ■ y is 
linear; 


If we think of vectors as points in- 
stead of arrows, then || a || should 
be interpreted as the distance from 
the origin to the point x. 



The length of this vector x is 
Jx 1 2 + x 2 2 . 


a • y = y ■ x for all a, y e R” . 
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An inner product is a generalization of the dot product. At this point you 
may be tempted to guess that an inner product is defined by abstracting the 
properties of the dot product discussed in the last paragraph. For real vector 
spaces, that guess is correct. However, so that we can make a definition that 
will be useful for both real and complex vector spaces, we need to examine 
the complex case before making the definition. 

Recall that if A = a + bi, where a, b € R, then 

• the absolute value of A, denoted |A|, is defined by |A| = J a 2 + b 2 \ 


• the complex conjugate of A, denoted A, is defined by A = a — bi ; 

• |A| 2 = AA. 


See Chapter 4 for the definitions and the basic properties of the absolute value 
and complex conjugate. 

For z = (z\ , . . . , z„) € C”, we define the norm of z by 



The absolute values are needed because we want ||z|| to be a nonnegative 
number. Note that 


z[| 2 = Zizi H b Z„z„. 


We want to think of ||z|| 2 as the inner product of z with itself, as we 
did in R' ! . The equation above thus suggests that the inner product of 
w = (w\, ... ,w n ) € C n with z should equal 


wfiZi + • • • T w n z n . 


If the roles of the w and z were interchanged, the expression above would 
be replaced with its complex conjugate. In other words, we should expect 
that the inner product of w with z equals the complex conjugate of the inner 
product of z with w. With that motivation, we are now ready to define an 
inner product on V, which may be a real or a complex vector space. 

Two comments about the notation used in the next definition: 

• If A is a complex number, then the notation A > 0 means that A is real 
and nonnegative. 

• We use the common notation ( u , v), with angle brackets denoting an 
inner product. Some people use parentheses instead, but then (u,v) 
becomes ambiguous because it could denote either an ordered pair or 
an inner product. 
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6.3 Definition inner product 

An inner product on V is a function that takes each ordered pair (u , v) of 
elements of V to a number (u, v) e F and has the following properties: 

positivity 

(v, v) > 0 for all v € V; 

definiteness 

(v, v) = 0 if and only if v = 0; 

additivity in first slot 

(u + v, w) = (w, w) + (v,w) for all u,v,w € V; 

homogeneity in first slot 

(An, v) = A (n, v) for all A e F and all n, v e V; 

conjugate symmetry 

(n, v) = (v, u) for all u, v € V. 


Although most mathematicians de- 
fine an inner product as above, 
many physicists use a definition 
that requires homogeneity in the 
second slot instead of the first slot. 


Every real number equals its com- 
plex conjugate. Thus if we are dealing 
with a real vector space, then in the last 
condition above we can dispense with 
the complex conjugate and simply state 
that (n, v) = (v, u) for all v,w € V. 


6.4 Example inner products 

(a) The Euclidean inner product on F”is defined by 

((Wl, . . - ,W„), (z 1, . . . ,Z„)) - WiZT H b W n zfi. 

(b) If ci , . . . , c„ are positive numbers, then an inner product can be defined 
on F” by 

{(w t, . . . ,w n ), (zi, . . . ,z„)> = ClWlZi H b c n w n zf. 

(c) An inner product can be defined on the vector space of continuous 
real- valued functions on the interval [—1, 1] by 

(fg) = J i f( x )g(x) dx. 

(d) An inner product can be defined on PfR) by 

/»oo 

{p,q) = / p(x)q(x)e~ x dx. 

Jo 
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6.5 Definition inner product space 

An inner product space is a vector space V along with an inner product 
on V. 

The most important example of an inner product space is F” with the 
Euclidean inner product given by part (a) of the last example. When F” is 
referred to as an inner product space, you should assume that the inner product 
is the Euclidean inner product unless explicitly told otherwise. 

So that we do not have to keep repeating the hypothesis that V is an inner 
product space, for the rest of this chapter we make the following assumption: 

6.6 Notation V 

For the rest of this chapter, V denotes an inner product space over F. 

Note the slight abuse of language here. An inner product space is a vector 
space along with an inner product on that vector space. When we say that 
a vector space V is an inner product space, we are also thinking that an 
inner product on V is lurking nearby or is obvious from the context (or is the 
Euclidean inner product if the vector space is F”). 

6.7 Basic properties of an inner product 

(a) For each fixed u e V, the function that takes v to (v, u) is a linear 
map from V to F. 

(b) (0, u) = 0 for every u e V. 

(c) {u. 0) = 0 for every u e V. 

(d) (w,v + w) = (u,v) + (u,w) for all u,v,w e V. 

(e) (w, Av) = A (u,v) for all A e F and u, v € V. 

Proof 

(a) Part (a) follows from the conditions of additivity in the first slot and 
homogeneity in the first slot in the definition of an inner product. 

(b) Part (b) follows from part (a) and the result that every linear map takes 
OtoO. 
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(c) Part (c) follows from part (a) and the conjugate symmetry property in 
the definition of an inner product. 

(d) Suppose u, v, w e V. Then 

(u, v + w) — (v + w,u) 

= (v, u) + (w, u) 

— (v, u) + (w, u) 

= (u,v) + (u,w). 

(e) Suppose A e F and u,v e V. Then 

(u, Av) = (Av, u) 

— A(v, u) 

= A(v, u) 

= A (m, v), 

as desired. ■ 

Norms 

Our motivation for defining inner products came initially from the norms of 
vectors on R 2 and R 3 . Now we see that each inner product determines a 
norm. 

6.8 Definition norm, ||v|| 

For v e V, the norm of v, denoted ||v||, is defined by 

|| v|| = y/ (v, v). 


6.9 Example norms 

(a) If ( 2:1 "n) S F" (with the Euclidean inner product), then 

\\(zi,...,Z n )\\ = y/\Zi \ 2 + ••• + |Z„| 2 . 


(b) In the vector space of continuous real-valued functions on [— 1 , 1] [with 
inner product given as in part (c) of 6.4], we have 
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6.10 Basic properties of the norm 
Suppose v e V. 

(a) || v|| = 0 if and only if v = 0. 

(b) || Av|| = | A | |M| for all A e F. 

Proof 

(a) The desired result holds because (v, v) = 0 if and only if v — 0. 

(b) Suppose A e F. Then 

|| Av|| 2 = (Av.Av) 

= A(v, Av) 

= AA(v, v) 

= |A| 2 ||v|| 2 . 

Taking square roots now gives the desired equality. ■ 

The proof above of part (b) illustrates a general principle: working with 
norms squared is usually easier than working directly with norms. 

Now we come to a crucial definition. 

6.11 Definition orthogonal 

Two vectors «,ve V are called orthogonal if (u,v) = 0. 

In the definition above, the order of the vectors does not matter, because 
(u,v) = 0 if and only if (v, u) — 0. Instead of saying that u and v are 
orthogonal, sometimes we say that u is orthogonal to v. 

Exercise 13 asks you to prove that if u, v are nonzero vectors in R 2 , then 

(m, v) = || r/ 1| ||v|| cos 6, 

where 9 is the angle between u and v (thinking of u and v as arrows with initial 
point at the origin). Thus two vectors in R 2 are orthogonal (with respect to the 
usual Euclidean inner product) if and only if the cosine of the angle between 
them is 0, which happens if and only if the vectors are perpendicular in the 
usual sense of plane geometry. Thus you can think of the word orthogonal as 
a fancy word meaning perpendicular. 
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We begin our study of orthogonality with an easy result. 

6.12 Orthogonality and 0 

(a) 0 is orthogonal to every vector in V. 

(b) 0 is the only vector in V that is orthogonal to itself. 

Proof 

(a) Part (b) of 6.7 states that (0, u) = 0 for every u € V. 

(b) If v e V and (v, v) = 0, then v = 0 (by definition of inner product). ■ 

For the special case V = R 2 , the 
next theorem is over 2,500 years old. 
Of course, the proof below is not the 
original proof. 

6.13 Pythagorean Theorem 

Suppose u and v are orthogonal vectors in V. Then 

II w + v|| 2 = ||u|| 2 + ||v|| 2 . 


The word orthogonal comes from 
the Greek word orthogonios, 
which means right-angled. 


Proof We have 


\u + v|| 


(u + v, u + v) 

(w,w) + (u,v) + (v, u) + (v, v) 

II || 2 2 

W + V , 


as desired. 


The proof given above of the 
Pythagorean Theorem shows that 
the conclusion holds if and only 
if (w,v) + {v,u), which equals 
2R e(w, v), is 0. Thus the converse 
of the Pythagorean Theorem holds 
in real inner product spaces. 


Suppose u, v e V, with v 0. We 
would like to write u as a scalar multiple 
of v plus a vector w orthogonal to v, as 
suggested in the next picture. 
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u 



An orthogonal decomposition. 

To discover how to write u as a scalar multiple of v plus a vector orthogonal 
to v, let c € F denote a scalar. Then 

u = cv + (u — cv ). 

Thus we need to choose c so that v is orthogonal to (u — cv). In other words, 
we want 

0 = (u — cv, v) = (u, v) — c||v|| 2 . 

The equation above shows that we should choose c to be (u, v)/||v|| 2 . Making 
this choice of c, we can write 



As you should verify, the equation above writes u as a scalar multiple of v 
plus a vector orthogonal to v. In other words, we have proved the following 
result. 

6.14 An orthogonal decomposition 
Suppose u, v € V, with v ^ 0. Set c = 

(w, v) = 0 and 

The orthogonal decomposition 6. 14 
will be used in the proof of the Cauchy- 
Schwarz Inequality, which is our next 
result and is one of the most important 
inequalities in mathematics. 


(m.v) , (k.v) 

— — and w = u — v. Then 

Wr V 2 

u = cv + w. 


French mathematician Augustin- 
Louis Cauchy ( 1789-1857 ) proved 
6.17(a) in 1821. German mathe- 
matician Hermann Schwarz (1843- 

1921) proved 6.17(b) in 1886. 
— 
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6.15 Cauchy-Schwarz Inequality 
Suppose u,v € V. Then 


|(w,v)| < || i/ 1| ||v||. 

This inequality is an equality if and only if one of u , v is a scalar multiple 
of the other. 


Proof If v = 0, then both sides of the desired inequality equal 0. Thus we 
can assume that v ^ 0. Consider the orthogonal decomposition 


w = 



v + w 


given by 6.14, where w is orthogonal to v. By the Pythagorean Theorem, 


(w, v) 


+ II w || 


I(K,V)| J 


+ H 


6.16 


> 


l(w,v)| 3 


Multiplying both sides of this inequality by ||v|| 2 and then taking square roots 
gives the desired inequality. 

Looking at the proof in the paragraph above, note that the Cauchy-Schwarz 
Inequality is an equality if and only if 6.16 is an equality. Obviously this 
happens if and only if w = 0. But w = 0 if and only if u is a multiple of v 
(see 6.14). Thus the Cauchy-Schwarz Inequality is an equality if and only if 
u is a scalar multiple of v or v is a scalar multiple of u (or both; the phrasing 
has been chosen to cover cases in which either u or v equals 0). ■ 


6.17 Example examples of the Cauchy-Schwarz Inequality 


(a) If x\ , . . . , x n , yi y„ e R, then 

\xiyi 4 h x n y n | 2 < (x\ 2 + 


+ x„ 2 )(yi 2 H h y„ 2 ). 


(b) If /, g are continuous real- valued functions on [— 1, 1], then 


/: 


f(x)g(x) dx 


< 


( J 1 (. fix )) 2 dx) ( J ^ (g(x)) 2 dx ) . 
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The next result, called the Triangle 
Inequality, has the geometric interpreta- 
tion that the length of each side of a tri- 
angle is less than the sum of the lengths 
of the other two sides. 

Note that the Triangle Inequality im- 
plies that the shortest path between two 
points is a line segment. 

6.18 Triangle Inequality 

Suppose u,v e V. Then 

|| u + v|| < ||m|| + || v|| . 

This inequality is an equality if and only if one of u , v is a nonnegative 
multiple of the other. 

Proof We have 

II w + v|| 2 = (u + V, u + v) 

= (u,u) + (v,v> + (u,v) + (v, u) 

— ( u,u ) + <v,v) + (m,v) + (w,v) 

= ||w|| 2 + ||v|| 2 + 2Re(u,v) 

6.19 < || w || 2 + ||v|| 2 + 2\(u, v)| 

6.20 < || w || 2 + || v || 2 + 2\\u\\ || v|| 

= (IMI + IMI) 2 , 

where 6.20 follows from the Cauchy-Schwarz Inequality (6.15). Taking 
square roots of both sides of the inequality above gives the desired inequality. 

The proof above shows that the Triangle Inequality is an equality if and 
only if we have equality in 6.19 and 6.20. Thus we have equality in the 
Triangle Inequality if and only if 

6.21 (w,v) = ||»|| || v|| . 

If one of u,v is a nonnegative multiple of the other, then 6.21 holds, as 
you should verify. Conversely, suppose 6.21 holds. Then the condition for 
equality in the Cauchy-Schwarz Inequality (6.15) implies that one of w, v is a 
scalar multiple of the other. Clearly 6.2 1 forces the scalar in question to be 
nonnegative, as desired. ■ 
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The next result is called the parallelogram equality because of its geometric 
interpretation: in every parallelogram, the sum of the squares of the lengths 
of the diagonals equals the sum of the squares of the lengths of the four sides. 


U 



The parallelogram equality. 


6.22 Parallelogram Equality 
Suppose u, v e V. Then 

\\u + v|| 2 + || u - v|| 2 = 2( || w || 2 + ||v|| 2 ). 

Proof We have 

|| w + r’ || 2 + ||m — r’|| 2 = (u + v, u + v) + (w — v, u — v) 

= IMI 2 + ll v l| 2 + (w.v) + ( v,u ) 

+ || w || 2 + || v|| 2 — (u, v) — (v, u) 

= 2(\\u\\ 2 + || v|| 2 ), 

as desired. ■ 


Law professor Richard Friedman presenting a case before the U.S. 
Supreme Court in 2010: 

Mr. Friedman: I think that issue is entirely orthogonal to the issue here 
because the Commonwealth is acknowledging — 

Chief Justice Roberts: I’m sorry. Entirely what? 

Mr. Friedman: Orthogonal. Right angle. Unrelated. Irrelevant. 

Chief Justice Roberts: Oh. 

Justice Scalia: What was that adjective? I liked that. 

Mr. Friedman: Orthogonal. 

Chief Justice Roberts: Orthogonal. 

Mr. Friedman: Right, right. 

Justice Scalia: Orthogonal, ooh. (Laughter.) 

Justice Kennedy: I knew this case presented us a problem. (Laughter.) 
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EXERCISES 6. A 


1 Show that the function that takes ((xi, X 2 ), (yi, J 2 )) € R 2 x R 2 to 
|xi_yi | + IX 2 J 2 I is not an inner product on R 2 . 

2 Show that the function that takes ((xi, X 2 , X 3 ), (yi, y 2 , y 3 )) e R 3 x R 3 
to Xi ji + X 3 V 3 is not an inner product on R 3 . 

3 Suppose F = R and V ^ {0}. Replace the positivity condition (which 
states that (v,v) > 0 for all v £ V) in the definition of an inner product 
(6.3) with the condition that <v,v> > 0 for some v £ V. Show that this 
change in the definition does not change the set of functions from V x V 
to R that are inner products on V. 

4 Suppose V is a real inner product space. 

(a) Show that (u + v, u — v) = ||w || 2 — ||v || 2 for every i/,ve V. 

(b) Show that if u, v € V have the same norm, then u +v is orthogonal 
to u — v. 

(c) Use part (b) to show that the diagonals of a rhombus are perpen- 


dicular to each other. 


5 Suppose T e C{V) is such that || 7"v|| < ||v|| for every v € V. Prove that 
T — y/2I is invertible. 

6 Suppose u, v e V. Prove that (u, v) — 0 if and only if 


u < m + av 


for all a e F. 

7 Suppose u, v £ V. Prove that \\au + bv\\ — \\bu + av || for all a,b £ R 
if and only if ||w|| = ||v||. 

8 Suppose u, v £ V and ||w|| = ||i’|| = 1 and (u, v) — 1. Prove that u — v. 

9 Suppose u, v £ V and ||m|| < 1 and ||v|| < 1. Prove that 



10 Find vectors u, v £ R 2 such that u is a scalar multiple of (1,3), v is 
orthogonal to ( 1 , 3), and ( 1 , 2 ) = u + v. 
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11 Prove that 

16 < (a -\- b -\- c d) ( — (- - — I — H — - j 
\a b c a ) 

for all positive numbers a, b,c,d. 

12 Prove that 

(x\ H b x n ) 2 < n(x i 2 4 b x n 2 ) 

for all positive integers n and all real numbers x\, ... ,x n . 

13 Suppose u, v are nonzero vectors in R 2 . Prove that 

(u, v) = ||u|| || v|| cos 9. 

where 6 is the angle between u and v (thinking of u and v as arrows with 
initial point at the origin). 

Hint: Draw the triangle formed by u, v, and u — v; then use the law of 
cosines. 


14 The angle between two vectors (thought of as arrows with initial point at 
the origin) in R 2 or R 3 can be defined geometrically. However, geometry 
is not as clear in R' 7 for n > 3. Thus the angle between two nonzero 
vectors x, y € R" is defined to be 


(x, 

x\\\\y\\ 


where the motivation for this definition comes from the previous exercise. 
Explain why the Cauchy-Schwarz Inequality is needed to show that this 
definition makes sense. 


15 Prove that 

n 9 n n , 2 

(E“A) 5(E«1(Ey) 

7=1 7=1 7=1 

for all real numbers a\, ... ,a n and b\ h„ . 

16 Suppose u, v € V are such that 

|| u || = 3, ||m + v||=4, || m — v|| = 6. 

What number does ||v|| equal? 
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17 Prove or disprove: there is an inner product on R 2 such that the associated 
norm is given by 

||(x, y)|| = max{x, y) 

for all (x, y) € R 2 . 

18 Suppose p > 0. Prove that there is an inner product on R 2 such that the 
associated norm is given by 

\\(x,y)\\ = (x p + y p ) l/p 

for all (x, y) € R 2 if and only if p = 2. 


19 Suppose V is a real inner product space. Prove that 

In + vll 2 — ||m — v II 2 


<n,v) 


for all u , v e V. 


20 Suppose V is a complex inner product space. Prove that 

, v \\u + v\\ 2 — \\u — v\\ 2 + \\u + iv\\ 2 i — \\u 

(n,v) = 

' 1 4 

for all u , v € V. 


iv\ri 


21 A norm on a vector space U is a function || || : U -a- [0, oo) such 
that ||n|| = 0 if and only if n = 0, ||an|| = |a|||w|| for all a e F 
and all u e U, and ||n + v|| < ||n|| + ||v|| for all n,v e U. Prove 
that a norm satisfying the parallelogram equality comes from an inner 
product (in other words, show that if || || is a norm on U satisfying the 
parallelogram equality, then there is an inner product ( , ) on U such 
that || m || = (n, u) 1 / 2 for all u e U). 

22 Show that the square of an average is less than or equal to the average 

of the squares. More precisely, show that if a\ a n e R, then the 

square of the average of a i , . . . , a n is less than or equal to the average 
of fli 2 , a n 2 . 


23 Suppose V \ , . . . , V m are inner product spaces. Show that the equation 
((i/l , . . . , Um)i (vi , . . . , V/jj)) = (u i , Vi) +••• + ( u m , v m ) 
defines an inner product on V\ x ■ ■ ■ x V m . 

[In the expression above on the right, {u\,v\) denotes the inner product 
onV i, . . . , (u m , v m ) denotes the inner product on V m . Each of the spaces 

V\ V m may have a different inner product, even though the same 

notation is used here. ] 
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24 Suppose S € £{V) is an injective operator on V. Define (•, -)i by 

(u,v) i = ( Su , Sv) 

for u, v € V. Show that (■, •) i is an inner product on V. 

25 Suppose S € C(V) is not injective. Define (•, ■) i as in the exercise above. 
Explain why (-, •) i is not an inner product on V. 

26 Suppose /, g are differentiable functions from R to R" 


(a) Show that 

(f(t)'g(t)}' = if H), git)) + (f(t),g'(t)). 

(b) Suppose c > 0 and II f(t)\\ — c for every t e R. Show that 
{fit), f{t)) = 0 for every t e R. 

(c) Interpret the result in part (b) geometrically in terms of the tangent 
vector to a curve lying on a sphere in R" centered at the origin. 


[For the exercise above , a function f : R — ^ R” is called differentiable 

if there exist differentiable functions f\ f n from R to R such that 

fit) = (flit), ■■■, fn(t)) for each t e R. Furthermore, for each t 6 R, 
the derivative fit) e R" is defined by fit) — (f\{t ), . . . , ff it)).) 


27 Suppose u,v,w e V. Prove that 

1 , m,2 ||W — U\f + \\W — V|| 

\\w-Uu + v)\\ 2 = 


\u — V 


28 Suppose C is a subset of V with the property that u,v e C implies 
^ (m + v) e C . Let w € V. Show that there is at most one point in C 
that is closest to w. In other words, show that there is at most one u e C 
such that 

|| w — w|| < || w — v|| forallveC. 

Flint: Use the previous exercise. 

29 For u,v e V, define d{u,v) = ||w — v||. 

(a) Show that d is a metric on V. 

(b) Show that if V is finite-dimensional, then d is a complete metric 
on V (meaning that every Cauchy sequence converges). 

(c) Show that every finite-dimensional subspace of V is a closed 
subset of V (with respect to the metric d). 
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30 


Fix a positive integer n. The Laplacian Ap of a twice differentiable 
function p on R' ? is the function on R” defined by 


Ap 


d 2 p 

dx 2 


+ ■•• + 


d 2 p 

9x2 ' 


The function p is called harmonic if Ap — 0. 


A polynomial on R' ! is a linear combination of functions of the 
form xi mi ■ ■ ■ x n m " , where mi, ... , m n are nonnegative integers. 

Suppose q is a polynomial on R". Prove that there exists a harmonic 
polynomial p on R" such that p (x ) = q(x) for every x e R” with 

[ The only fact about harmonic functions that you need for this exercise 
is that if p is a harmonic function on R” and p(x) = 0 for all x € R” 
with ||*|| = 1 , then p = 0 .] 


Hint: A reasonable guess is that the desired harmonic polynomial p is of 
the form q + (1 — ||x|| 2 )r for some polynomial r. Prove that there is a 
polynomial r on R” such that q + (1 — ||x|| 2 )r is harmonic by defining 
an operator T on a suitable vector space by 

Tr = A((l - || x || 2 )r) 

and then showing that T is injective and hence surjective. 


31 Use inner products to prove Apollonius’s Identity: In a triangle with 
sides of length a, b, and c, let d be the length of the line segment from 
the midpoint of the side of length c to the opposite vertex. Then 

a 2 + b 2 — \c 2 + 2d 2 . 
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6.B 


Orthonormal Bases 


6.23 Definition orthonormal 

• A list of vectors is called orthonormal if each vector in the list has 
norm 1 and is orthogonal to all the other vectors in the list. 

• In other words, a list e \ , . . . , e m of vectors in V is orthonormal if 


{ej,e k ) 


( 1 if j = k, 
(0 if j ± k. 


6.24 Example orthonormal lists 

(a) The standard basis in F" is an orthonormal list. 

(b) ( 7f ’ 7f ’ ) ’ ( _ 71 ’ • °) is an orthonormal list in F 3 . 

(c) <71' ■ 71’ 7?)’ (- 72 ’ TI’ 0 )’ 7T "^) is an orthonormal list 

in F 3 . 

Orthonormal lists are particularly easy to work with, as illustrated by the 
next result. 

6.25 The norm of an orthonormal linear combination 
If e \ , . . . , e m is an orthonormal list of vectors in V, then 

|| <2 \e\ + • • • + a m e m II 2 = | fl t| 2 + ■ ■ ■ + \a m \~ 
for all a 1 a m e F. 

Proof Because each ej has norm 1, this follows easily from repeated appli- 
cations of the Pythagorean Theorem (6.13). ■ 

The result above has the following important corollary. 

6.26 An orthonormal list is linearly independent 
Every orthonormal list of vectors is linearly independent. 
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Proof Suppose e\,...,e m is an orthonormal list of vectors in V and 
a i , . . . , a m e F are such that 

a i£>i H h a m e m = 0. 

Then |<3i| 2 + • • • + \a m \ 2 = 0 (by 6.25), which means that all the af s are 0. 
Thus ei, ... ,e m is linearly independent. ■ 

6.27 Definition orthonormal basis 

An orthonormal basis of V is an orthonormal list of vectors in V that is 
also a basis of V. 

For example, the standard basis is an orthonormal basis of F”. 

6.28 An orthonormal list of the right length is an orthonormal basis 

Every orthonormal list of vectors in V with length dim V is an orthonormal 
basis of V. 

Proof By 6.26, any such list must be linearly independent; because it has the 
right length, it is a basis — see 2.39. ■ 


6.29 Example Show that 

(i t_ t lwi l _J_ _i\ _i _i j_\ /_i i _i t\ 
\2 ’ 2 ’ 2 ’ 2 / ’ \2 ’ 2 ’ 2’ 2)’V2’ 2’ 2’2)’V 2 ’ 2 > 2’2l 

is an orthonormal basis of F 4 . 


Solution We have 

i ik’ 5) 1 = V (i ) 2 + (i ) 2 + = L 

Similarly, the other three vectors in the list above also have norm 1. 
We have 


III 1 1 1) (1 1 -1 -l)\ =1.1 + 1. 1 , 1 . / — J_\ , 1 . /_ i \ = o 

\V2 ’ 2 ’ 2 ’ 2/ ’ \2 ’ 2 ’ 2’ 2 II 2 2^2 2^2 \ 2) ~ 2 \ 2) yJ ' 


Similarly, the inner product of any two distinct vectors in the list above also 
equals 0. 

Thus the list above is orthonormal. Because we have an orthonormal list of 
length four in the four-dimensional vector space F 4 , this list is an orthonormal 
basis of F 4 (by 6.28). 
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In general, given a basis e^, . . . , e„ of V and a vector v € V, we know that 
there is some choice of scalars a i a n e F such that 


v — a\e\ + • • • + a n e n . 

Computing the numbers a\, ... ,a n that 
satisfy the equation above can be diffi- 
cult for an arbitrary basis of V. The 
next result shows, however, that this is 
easy for an orthonormal basis — just take 
a ,i = ( v,ej ). 

6.30 Writing a vector as linear combination of orthonormal basis 
Suppose e \ , . . . , e n is an orthonormal basis of V and v € V. Then 


The importance of orthonormal 
bases stems mainly from the next 
result. 


v = (v,ei)<?i H b (v, e n )e n 


and 

IMI 2 = |(v,ei)| 2 + ••■ + |(v, e n )\ 2 . 

Proof Because e\. ... ,e n is a basis of V, there exist scalars a \ , . . . , a n such 
that 

v = ci\e\ + ••• + a n e n . 

Because e \ , . . . , e n is orthonormal, taking the inner product of both sides of 
this equation with ej gives (v, ej) — aj. Thus the first equation in 6.30 holds. 

The second equation in 6.30 follows immediately from the first equation 
and 6.25. ■ 

Now that we understand the usefulness of orthonormal bases, how do we 
go about finding them? For example, does V m (R), with inner product given 
by integration on [—1, 1] [see 6.4(c)], have an orthonormal basis? The next 
result will lead to answers to these questions. 

The algorithm used in the next proof 
is called the Gram-Schmidt Procedure . 
It gives a method for turning a linearly 
independent list into an orthonormal list 
with the same span as the original list. 


Danish mathematician J0rgen 
Gram (1850-1916) and German 
mathematician Erhard Schmidt 
(1876-1959) popularized this algo- 
rithm that constructs orthonormal 
lists. 
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6.31 Gram-Schmidt Procedure 

Suppose t'i , . . . , v m is a linearly independent list of vectors in V. Let 
ei = v i / 1 1 v 1 1| . For j = 2, . . . , m, define ej inductively by 

e . _ v i ~ ( v L g i)ei {vj^j-i)ej-i 

llw - (v jt e i)ei (vj,ej-i)ej-i\\' 

Then e\, ... ,e m is an orthonormal list of vectors in V such that 


span(vi, . . ,,vj) = span(ei, . . ,,ej) 


for j = 1, .... m. 

Proof We will show by induction on j that the desired conclusion holds. To 
get started with j — 1, note that span (vi) = span(ci) because vi is a positive 
multiple of e\. 

Suppose 1 < j < m and we have verified that 

6.32 span(vi, . . . ,v 7 _i) = span(ei e/-i). 

Note that vj £ span(vi ..... v 7 _i) (because vi , . . . , v m is linearly indepen- 
dent). Thus Vj span(ci ..... c 7 -i )■ Hence we are not dividing by 0 in the 
definition of e 7 given in 6.31. Dividing a vector by its norm produces a new 
vector with norm 1; thus ||e 7 || = 1. 

Let 1 < k < j . Then 

vj ~ (vj,e i)ei (vj^ej-i) 

II vj ~ {vj,e i)ei {vj,ej- 1 ) 

(vj^k) - (vj,e k ) 

vj - ( Vj ,e i)ei (vj ,ej-\)ej-i || 

= 0. 

Thus e\ , ,ej is an orthonormal list. 

From the definition of ej given in 6.31, we see that Vj e span(ei, .... ej). 
Combining this information with 6.32 shows that 

span(vi, .... Vj) C span(ci, .... ej). 

Both lists above are linearly independent (the v’s by hypothesis, the <?’s by 
orthonormality and 6.26). Thus both subspaces above have dimension j , and 
hence they are equal, completing the proof. ■ 
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6.33 Example Find an orthonormal basis of V 2 (R), where the inner prod- 
uct is given by (p, q) — p(x)q(x) dx. 

Solution We will apply the Gram-Schmidt Procedure (6.31) to the basis 

l,x,x 2 . 

To get started, with this inner product we have 

|| 1|| 2 = J l 2 dx = 2. 


Thus || 1 1| = a/2, and hence e\ = Jj. 

Now the numerator in the expression for e 2 is 


x — (x, e\)e\ = x 


(j_ x x J\ dx )'Jh 


= x. 


We have 


/: 


\\x\\ 2 = I x 2 dx = |. 


Thus || jc || = J |, and hence 62 = y \x. 

Now the numerator in the expression for ej, is 


x 2 - (x 2 , ei>£?i - (x 2 , e 2 )e 2 

= ^-(L/Jidx )^- ( J ^x 2 



We have 


Thus || x 2 
Thus 




|x 2 + |) dx 


_ 8 _ 

45' 


III 
3 II 



and hence e 2 



j_ 

3 


)■ 



1 

3 


) 


is an orthonormal list of length 3 in V 2 {R.). Hence this orthonormal list is an 
orthonormal basis of T 2 (R.) by 6.28. 
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Now we can answer the question about the existence of orthonormal bases. 

6.34 Existence of orthonormal basis 

Every finite-dimensional inner product space has an orthonormal basis. 

Proof Suppose V is finite-dimensional. Choose a basis of V. Apply the 
Gram-Schmidt Procedure (6.31) to it, producing an orthonormal list with 
length dim V. By 6.28, this orthonormal list is an orthonormal basis of V. m 

Sometimes we need to know not only that an orthonormal basis exists, but 
also that every orthonormal list can be extended to an orthonormal basis. In 
the next corollary, the Gram-Schmidt Procedure shows that such an extension 
is always possible. 

6.35 Orthonormal list extends to orthonormal basis 

Suppose V is finite-dimensional. Then every orthonormal list of vectors 
in V can be extended to an orthonormal basis of V. 

Proof Suppose ei, ... ,e m is an orthonormal list of vectors in V. Then 
e \ , . . . , e m is linearly independent (by 6.26). Hence this list can be extended to 
a basis e \ , . . . , e m , iq , . . . , v n of V (see 2.33). Now apply the Gram-Schmidt 
Procedure (6.31) to e\, . . . , e m , vi, . . . , v„, producing an orthonormal list 


6.36 


^1 > • • • i f \ ) • • • > fn\ 


here the formula given by the Gram-Schmidt Procedure leaves the first m 
vectors unchanged because they are already orthonormal. The list above is an 


orthonormal basis of V by 6.28. 


Recall that a matrix is called upper triangular if all entries below the 
diagonal equal 0. In other words, an upper-triangular matrix looks like this: 

\ 





where the 0 in the matrix above indicates that all entries below the diagonal 
equal 0, and asterisks are used to denote entries on and above the diagonal. 
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In the last chapter we showed that if V is a finite-dimensional complex 
vector space, then for each operator on V there is a basis with respect to 
which the matrix of the operator is upper triangular (see 5.27). Now that we 
are dealing with inner product spaces, we would like to know whether there 
exists an orthonormal basis with respect to which we have an upper-triangular 
matrix. 

The next result shows that the existence of a basis with respect to which 
T has an upper-triangular matrix implies the existence of an orthonormal 
basis with this property. This result is true on both real and complex vector 
spaces (although on a real vector space, the hypothesis holds only for some 
operators). 

6.37 Upper-triangular matrix with respect to orthonormal basis 

Suppose T e C(V). If T has an upper-triangular matrix with respect to 
some basis of V, then T has an upper-triangular matrix with respect to 
some orthonormal basis of V. 

Proof Suppose T has an upper-triangular matrix with respect to some basis 
vi, . . . , v n of V. Thus spanfi’i , . . . , v 7 ) is invariant under T for each j — 
1, . . . , 77 (see 5.26). 

Apply the Gram-Schmidt Procedure to vi, . . . , v„, producing an orthonor- 
mal basis e\, ... ,e n of V. Because 

span(ei ,...,ej) — spanfvi , . . . , v 7 ) 

for each j (see 6.31), we conclude that spanfe j , . . . , <? 7 ) is invariant under T 
for each j = 1 Thus, by 5.26, T has an upper-triangular matrix with 
respect to the orthonormal basis e\, ... ,e n . ■ 

The next result is an important appli- 
cation of the result above. 


German mathematician Issai Schur 
(1875-1941) published the first 
proof of the next result in 1909. 


6.38 Schur’s Theorem 

Suppose V is a finite-dimensional complex vector space and T e C(V). 
Then T has an upper-triangular matrix with respect to some orthonormal 
basis of V. 

Proof Recall that T has an upper-triangular matrix with respect to some basis 
of V (see 5.27). Now apply 6.37. ■ 
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Linear Functionals on Inner Product Spaces 

Because linear maps into the scalar field F play a special role, we defined a 
special name for them in Section 3.F. That definition is repeated below in 
case you skipped Section 3.F. 

6.39 Definition linear functional 

A linear functional on V is a linear map from V to F. In other words, a 
linear functional is an element of C( V, F). 

6.40 Example The function cp : F 3 — »■ F defined by 
<p(zi,z 2 ,z 3 ) = 2zi - 5z 2 + z 3 

is a linear functional on F 3 . We could write this linear functional in the form 

(p(z) — (z, u) 

for every z € F 3 , where u = (2, —5, 1). 


6.41 Example The function (p : ^(R) — > R defined by 
<P(P) = /-, />(t)(cOS(7T?)) dt 

is a linear functional on T^fR) (here the inner product on 'P 2 (R) is multi- 
plication followed by integration on [—1, 1]; see 6.33). It is not obvious that 
there exists u e 'P 2 (R) such that 

9(P) = (p,m) 

for every p e 'P 2 (R) [we cannot take u (l ) = cos(jtI) because that function 
is not an element of T* 2 (R)]. 


If u e V, then the map that sends 
v to (v, u) is a linear functional on V. 
The next result shows that every linear 
functional on V is of this form. Ex- 
ample 6.41 above illustrates the power 
of the next result because for the linear 
functional in that example, there is no 
obvious candidate for u. 


The next result is named in honor of 
Hungarian mathematician Frigyes 
Riesz ( 1880-1956 ), who proved 
several results early in the twen- 
tieth century that look very much 
like the result below. 
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6.42 Riesz Representation Theorem 

Suppose V is finite-dimensional and <p is a linear functional on V. Then 
there is a unique vector u € V such that 

<P(v) = (v,w) 


for every v € V. 

Proof First we show there exists a vector u € V such that <p(v) = (v, u) for 
every v e V. Let e \ , . . . , e n be an orthonormal basis of V. Then 

<p(v) = <p{(v,e \)e\ H P {v,e„)e n ) 

= (v,ei)<p(ei) -\ F (v, e n )(p(e n ) 

= {v,<p{e{)ei H h (p(e n )e n ) 

for every v e V, where the first equality comes from 6.30. Thus setting 

6.43 u - (p(e i)e\ -\ h cp(e n )e n , 

we have cp(v) = {v. u) for every v e V, as desired. 

Now we prove that only one vector u € V has the desired behavior. 
Suppose Mi, U2 e V are such that 

(p(v) = (v, Ml) = (v,U 2 ) 


for every v € V. Then 

0 = (v, Ml) - (v, M 2 > = (v, MI - M 2 ) 

for every v e V. Taking v = u\—u 2 shows that ii \ —u 2 = 0 . In other words, 
Mi = u 2 , completing the proof of the uniqueness part of the result. ■ 


6.44 Example Find m e V 2 (R) such that 


J p(t)(cos(nt)) dt = J 


p(t)u(t) dt 


for every p e V 2 (R.). 
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Solution Let <p(p) = /i x p(t)(cos(nt)) dt. Applying formula 6.43 from 
the proof above, and using the orthonormal basis from Example 6.33, we have 

u(x) — (^J y^(cOS(7Tf)) dt^j + {^j yJ~^t(cOS(7Tt)) dt^j 

+ ( j _ |)(cos(;rf)) dt} — !)• 

A bit of calculus shows that 

m(x) = -^(x 1 2 -|). 


Suppose V is finite-dimensional and cp a linear functional on V. Then 6.43 
gives a formula for the vector u that satisfies (p(v) — ( v , u) for all v e V. 
Specifically, we have 


u = <p(e i)ei 4 b <p(e n )e n . 

The right side of the equation above seems to depend on the orthonormal 
basis e\, . . . ,e n as well as on <p. However, 6.42 tells us that u is uniquely 
determined by (p. Thus the right side of the equation above is the same 
regardless of which orthonormal basis e \ , . . . , e n of V is chosen. 

EXERCISES 6.B 


1 (a) Suppose 6 e R. Show that (cos 0, sin 6), (—sin 9, cos 9) and 

(cos 9, sin 6), (sin 9 , — cos 9) are orthonormal bases of R 2 . 

(b) Show that each orthonormal basis of R 2 is of the form given by 
one of the two possibilities of part (a). 

2 Suppose e\, ... ,e m is an orthonormal list of vectors in V. Let v e V. 
Prove that 

||v|| 2 = |(v,e 1 >| 2 + --- + |(v,e m )| 2 
if and only if v € span(ei , ... ,e m ). 

3 Suppose T e £(R 3 ) has an upper-triangular matrix with respect to 
the basis (1, 0, 0), (1, 1, 1), (1, 1,2). Find an orthonormal basis of R 3 
(use the usual inner product on R 3 ) with respect to which T has an 
upper-triangular matrix. 
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4 Suppose n is a positive integer. Prove that 


cosx cos2x cos nx sinx sin2x 


stnnx 



is an orthonormal list of vectors in C [—n, n\, the vector space of contin- 
uous real-valued functions on [— n. n] with inner product 



[The orThonormal list above is often used for modeling periodic phenom- 
ena such as tides.] 

5 On TMRK consider the inner product given by 



Apply the Gram-Schmidt Procedure to the basis 1, x, x 2 to produce an 
orthonormal basis of 7^2 ( R ) ■ 

6 Find an orthonormal basis of V 2 (R) (with inner product as in Exercise 5) 
such that the differentiation operator (the operator that takes p to p') 
on -P 2 (R) has an upper-triangular matrix with respect to this basis. 

7 Find a polynomial q e 7^2 (R) such that 



for every p e TMR)- 


8 Find a polynomial q e V 2 (R) such that 


/ p(x)(cos nx) dx — / p(x)q(x)dx 

Jo Jo 


9 


for every p e ^(R)- 

What happens if the Gram-Schmidt Procedure is applied to a list of 
vectors that is not linearly independent? 


SECTION 6.B Orthonormal Bases 


191 


10 Suppose V is a real inner product space and v\ , . . . , v m is a linearly inde- 
pendent list of vectors in V. Prove that there exist exactly 2 m orthonormal 
lists e\ Cm of vectors in V such that 


span(vi Vj) = span(ci ej ) 


for all j e {1, . . . , m}. 

11 Suppose (•, •) i and (-, -)2 are inner products on V such that (v, w) i = 0 
if and only if (v, w) 2 = 0. Prove that there is a positive number c such 
that (v, w)i = c(v,w ) 2 for every v,w € V. 

12 Suppose V is finite-dimensional and i , {-,-)2 are inner products on 

V with corresponding norms || ■ || i and || • || 2 - Prove that there exists a 
positive number c such that 


v||i < c||v|| 2 


for every v e V. 

13 Suppose vi , . . . , v m is a linearly independent list in V. Show that there 
exists w € V such that (w, vj) >0 for all j € {1 , ,m). 

14 Suppose e\, . . . ,e n is an orthonormal basis of V and vi, . . . , v n are 
vectors in V such that 

" ej - v jII< 7h 

for each j . Prove that vi , . . . , v n is a basis of V. 

15 Suppose Cr([— 1, 1]) is the vector space of continuous real-valued func- 
tions on the interval [—1, 1] with inner product given by 



for fg e Cr([— 1, 1]). Let ip be the linear functional on Cr([— L 1]) 
defined by (p( f) = /( 0). Show that there does not exist g e Cr([— L 1]) 
such that 


<P(f) = (fg) 


for every / e C R ([-1, 1]). 

[The exercise above shows that the Riesz Representation Theorem (6.42) 
does not hold on infinite-dimensional vector spaces without additional 
hypotheses on V and <p. \ 
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16 Suppose F = C, V is finite-dimensional, T e C( V), all the eigenvalues 
of T have absolute value less than 1 , and e > 0. Prove that there exists a 
positive integer m such that || T m v\\ < e||v|| for every v € V. 

17 For u € V, let ( Pu denote the linear functional on V defined by 

(d>w)(v) = (v,u) 


for v € V. 

(a) Show that if F = R, then <3> is a linear map from V to V. (Recall 
from Section 3.F that V' — C(V. F) and that V is called the dual 
space of V.) 

(b) Show that if F = C and V ^ {0}, then <f> is not a linear map. 

(c) Show that is injective. 

(d) Suppose F = R and V is finite-dimensional. Use parts (a) and (c) 
and a dimension-counting argument (but without using 6.42) to 
show that <P is an isomorphism from V onto V. 

[Part (d) gives an alternative proof of the Riesz Representation Theorem 
(6.42) when F = R. Part (d) also gives a natural isomorphism (meaning 
that it does not depend on a choice of basis) from a finite-dimensional 
real inner product space onto its dual space.] 
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Orthogonal Complements and 
Minimization Problems 

Orthogonal Complements 

6.45 Definition orthogonal complement, C/ 3- 

If U is a subset of V, then the orthogonal complement of U, denoted U^, 
is the set of all vectors in V that are orthogonal to every vector in U : 

U' L = {v e V : (v, u) = 0 for every u € U }. 

For example, if U is a line in R 3 , then U 1 - is the plane containing the 
origin that is perpendicular to U. If U is a plane in R 3 , then U 1 - is the line 
containing the origin that is perpendicular to U. 

6.46 Basic properties of orthogonal complement 

(a) If U is a subset of V, then U 1 - is a subspace of V. 

(b) {0} 3- = V. 

(c) V 1 - = {0}. 

(d) If U is a subset of V, then U D U 1 - C {0}. 

(e) If U and W are subsets of V and U C W, then W 1 - C U 

Proof 

(a) Suppose U is a subset of V. Then (0, u) = 0 for every u e U\ thus 
0 € U 1 -. 

Suppose v, w € [/-*-. If u € U, then 

(v + w,u) = { v , it) + (w,u) = 0 + 0 = 0. 

Thus v + w 6 t/ 1 . In other words, is closed under addition. 
Similarly, suppose A e F and v e U ^ . If u e U, then 

(Av, u) — A(v, u) = A • 0 = 0. 

Thus Av e . In other words, U 1 - is closed under scalar multiplica- 
tion. Thus U 1 - is a subspace of V. 


6.C 
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(b) Suppose v € V. Then (v, 0) = 0, which implies that v € {0}- 1 -. Thus 
{0}^ = V. 

(c) Suppose v e V Then (v, v) = 0, which implies that v = 0. Thus 

V 1 = { 0 }. 

(d) Suppose U is a subset of V and vel/fl (/-*-. Then (v, v) = 0, which 
implies that v = 0. Thus U D U 1 - C {0}. 

(e) Suppose U and W are subsets of V and U C W. Suppose v € W 1 - . 

Then (v, u) = 0 for every u € W, which implies that (v, u) = 0 for 
every u e U. Hence v e U Thus W 1 - C U^. m 

Recall that if U, W are subspaces of V, then V is the direct sum of U and 
W (written V = U © W) if each element of V can be written in exactly one 
way as a vector in U plus a vector in W (see 1.40). 

The next result shows that every finite-dimensional subspace of V leads to 
a natural direct sum decomposition of V. 

6.47 Direct sum of a subspace and its orthogonal complement 
Suppose U is a finite-dimensional subspace of V. Then 

V = U © U ± . 


Proof First we will show that 

6.48 V = U + U ± . 

To do this, suppose v € V. Let ei, . . . ,e m be an orthonormal basis of U. 
Obviously 

6.49 v = (v, e\)e\ 4 F ( v,e m )e m +v - (v,ei)ei {v,e m )e m . 

v -j -j 

v 

U W 

Let u and w be defined as in the equation above. Clearly u e U. Because 
e\ , . . . , e m is an orthonormal list, for each j = 1 , . . . , m we have 

{w,ej) = {v,ej) - (v, ej) 

= 0. 

Thus w is orthogonal to every vector in span(<?i , . . . , e m ). In other words, 
w e U^. Thus we have written v = u + w, where u e U and w e U ■*", 
completing the proof of 6.48. 

From 6.46(d), we know that U n U 1 - — {0}. Along with 6.48, this implies 
that V = U © U 1 - (see 1.45). ■ 
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Now we can see how to compute dim U 1 - from dim U. 

6.50 Dimension of the orthogonal complement 
Suppose V is finite-dimensional and U is a subspace of V. Then 

dim U ± = dim V — dim U. 

Proof The formula for dim U 1 - follows immediately from 6.47 and 3.78. ■ 
The next result is an important consequence of 6.47. 

6.51 The orthogonal complement of the orthogonal complement 
Suppose U is a finite-dimensional subspace of V. Then 

U = (i u ± )- L . 


Proof First we will show that 

6.52 U C (f7 ± ) ± . 

To do this, suppose u e U. Then (u,v) = 0 for every v e U 1 - (by the 
definition of U^). Because u is orthogonal to every vector in U^, we have 
u e (U ■ L ) _L , completing the proof of 6.52. 

To prove the inclusion in the other direction, suppose v € (fy- 1 -)- 1 -. By 
6.47, we can write v = u + w, where u € U and we t/ _L . We have 
v — u = w e U 1 - . Because v e (T/ -1 - )-*- and u e ( t/- 1 -)- 1 - (from 6.52), we 
have v — u e (C/ _L )' L . Thus v — u e U 1 - Cl ((y- 1 -)- 1 -, which implies that v — u 
is orthogonal to itself, which implies that v — u — 0, which implies that 
v = u, which implies that v e U. Thus (L"'^)^ C U, which along with 6.52 
completes the proof. ■ 

We now define an operator Vu for each finite-dimensional subspace of V. 
6.53 Definition orthogonal projection, Pjj 

Suppose U is a finite-dimensional subspace of V. The orthogonal 
projection of V onto U is the operator Pjj e C( V) defined as follows: 
For v e V, write v = u + w, where u e U and w e U^. Then Pjjv = u. 
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The direct sum decomposition V — U © U 1 - given by 6.47 shows that 
each v € V can be uniquely written in the form v = u + w with u € U and 
w € [/-*-. Thus Pjjv is well defined. 


6.54 Example Suppose x € V with r / 0 and U — span(x). Show that 



for every v € V. 

Solution Suppose v € V. Then 



where the first term on the right is in span(x) (and thus in U) and the second 
term on the right is orthogonal to x (and thus is in £/-*-). Thus Pj/v equals the 
first term on the right, as desired. 

6.55 Properties of the orthogonal projection P v 

Suppose U is a finite-dimensional subspace of V and ref, Then 

(a) P v e C{V) ; 

(b) Pjju = u for every u e U\ 

(c) Pjj w = 0 for every vv e U x ; 

(d) range Pjj = U\ 

(e) null Pu = [/-*-; 

(f) v-PjjveU - 1 -; 

(g) Pu 2 = Pu ; 

(h) \\P UV \\ < ||t’|| ; 

(i) for every orthonormal basis e\, ... ,e m of U, 


Puv = (v, e\)e\ 4 h {v.e m )e m . 
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Proof 

(a) To show that Py is a linear map on V, suppose vi , V 2 e V. Write 

v\ — u\ + w\ and V 2 = 112 + W 2 

with U\,U 2 £ U andwi,W 2 € U^. Thus Pyv \ — n 1 and Pjjv 2 = U 2 . 
Now 

V 1 + V 2 = (Ml + M2) + (Wl + W2), 

where mi + M 2 e £/ and wi + W 2 e f/- 1 -. Thus 

Pt/(vi + v 2 ) = Mi + m 2 = Ptm + Pt/v 2 . 

Similarly, suppose A e F. The equation v = n + w with u e U and 
w e U 1 - implies that Av = Am + Aw with Am e U and Aw e U 1 - . 
Thus Pu( Av) = Am = XPjjv. 

Hence P(j is a linear map from V to V. 

(b) Suppose u e U. We can write u = n + 0, where u e U and 0 e U ^ . 
Thus Pf/M = M. 

(c) Suppose w e U We can write w = 0 + w, where 0 e U and w e P -1 -. 
Thus Pc/w = 0. 

(d) The definition of Pu implies that range Pu C U. Part (b) implies that 
U C range Py. Thus range Py — U. 

(e) Part (c) implies that U 1 - C null Py. To prove the inclusion in the other 
direction, note that if v e null Py then the decomposition given by 6.47 
must be v = 0 + v, where 0 e U and v e U J -. Thus null Py C U ■ L . 

(f) If v = m + w with u e U and w e f/ -1 -, then 

v — P(/v = v — u = w S P -1- . 

(g) If v = m + w with u € U and w e t/ -1 -, then 

(Pu 2 )v = Pu(Puv) = Puu — u — Pyv. 

(h) If v = m + w with u € U and w e then 

ll-Pcvll 2 = 1 1 m 1 1 2 < ||m|| 2 + 1 1 w 1 1 2 = || v|| 2 , 
where the last equality comes from the Pythagorean Theorem. 

(i) The formula for Pyv follows from equation 6.49 in the proof of 6.47. ■ 
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Minimization Problems 

The following problem often arises: 
given a subspace U of V and a point 
v € V, find a point u e U such that 
|| v — w|| is as small as possible. The 
next proposition shows that this mini- 
mization problem is solved by taking 
u = Puv. 


The remarkable simplicity of the so- 
lution to this minimization problem 
has led to many important applica- 
tions of inner product spaces out- 
side of pure mathematics. 


6.56 Minimizing the distance to a subspace 

Suppose U is a finite-dimensional subspace of V, v e V, and u e U. Then 

|| v — Pjjv || < || v — m||. 

Furthermore, the inequality above is an equality if and only if u = Pjjv. 


Proof We have 


6.57 || v — P v v\\ 2 < || v — Pjjv || 2 + \\Puv — u\\ 2 

= II (v — P v v) + (Puv-u) || 2 


where the first line above holds because 0 < || Pjjv — u|| 2 , the second 
line above comes from the Pythagorean Theorem [which applies because 
v — Pjjv € U 1 - by 6.55(f), and Pjjv — u € U], and the third line above holds 
by simple algebra. Taking square roots gives the desired inequality. 

Our inequality above is an equality if and only if 6.57 is an equality, 
which happens if and only if || Py v — u || =0, which happens if and only if 
u — Puv. m 



Puv is the closest point in U to v. 
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The last result is often combined with the formula 6.55(i) to compute 
explicit solutions to minimization problems. 


6.58 Example Find a polynomial u with real coefficients and degree at 
most 5 that approximates sin x as well as possible on the interval \—n, n], in 
the sense that 



is as small as possible. Compare this result to the Taylor series approximation. 

Solution Let Cr[— jz, n] denote the real inner product space of continuous 
real-valued functions on [— n, n] with inner product 



6.59 


Let v e Cr [ — jt. n\ be the function defined by v(x) = sinx. Let U denote the 
subspace of Cr[— Jt, n] consisting of the polynomials with real coefficients 
and degree at most 5. Our problem can now be reformulated as follows: 


Find u € U such that ||v — u|| is as small as possible. 


To compute the solution to our ap- ^ computer that can perform inte - 1 
proximation problem, first apply the gra tions is useful here. 
Gram-Schmidt Procedure (using the in- 
ner product given by 6.59) to the basis 1, x, x 2 , x 3 , x 4 , x 5 of U, producing 
an orthonormal basis e\, ei, £3,^4, 65, of U. Then, again using the inner 
product given by 6.59, compute Pjjv using 6.55(i) (with m = 6). Doing this 
computation shows that Pjjv is the function u defined by 

6.60 w(x) = 0.987862x - 0.155271x 3 + 0.00564312x 5 , 

where the n ’s that appear in the exact answer have been replaced with a good 
decimal approximation. 

By 6.56, the polynomial u above is the best approximation to sinx on 
[— n, n] using polynomials of degree at most 5 (here “best approximation” 
means in the sense of minimizing \ sinx — u(x) \ 2 dx). To see how good 

this approximation is, the next figure shows the graphs of both sinx and our 
approximation u(x) given by 6.60 over the interval [—n, re]. 
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Graphs on [~tc. n] of sinx (blue) and 
its approximation u(x) (red) given by 6.60. 


Our approximation 6.60 is so accurate that the two graphs are almost 
identical — our eyes may see only one graph! Here the blue graph is placed 
almost exactly over the red graph. If you are viewing this on an electronic 
device, try enlarging the picture above, especially near 3 or —3, to see a small 
gap between the two graphs. 

Another well-known approximation to sin x by a polynomial of degree 5 
is given by the Taylor polynomial 


6.61 


3 S 

X X 

x 1 . 

3! 5! 


To see how good this approximation is, the next picture shows the graphs of 
both sinx and the Taylor polynomial 6.61 over the interval [— n, n]. 



Graphs on [—n. n] of sinx (blue) and the Taylor polynomial 6.61 (red). 


The Taylor polynomial is an excellent approximation to sin x for x near 0. 
But the picture above shows that for |x| > 2, the Taylor polynomial is not 
so accurate, especially compared to 6.60. For example, taking x = 3, our 
approximation 6.60 estimates sin 3 with an error of about 0.001, but the Taylor 
series 6.61 estimates sin 3 with an error of about 0.4. Thus at x = 3, the error 
in the Taylor series is hundreds of times larger than the error given by 6.60. 
Linear algebra has helped us discover an approximation to sin x that improves 
upon what we learned in calculus! 
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EXERCISES 6.C 


1 Suppose v\, , v m e V. Prove that 

{vi, ■ ■ .Tm} 1 = (span(vi, . . . ,t ’m)) 1 ■ 

2 Suppose U is a finite-dimensional subspace of V. Prove that U 1 - = {0} 
if and only if U — V. 

[Exercise 14(a) shows that the result above is not true without the hy- 
pothesis that U is finite-dimensional] 

3 Suppose U is a subspace of V with basis u\,...,u m and 

. . . , . . . ,Wyi 

is a basis of V. Prove that if the Gram-Schmidt Procedure is applied 
to the basis of V above, producing a list e\, . . . , e m , fi, . . . , f n , then 
e\ , . . . , e m is an orthonormal basis of U and f\ , . . . , f n is an orthonor- 
mal basis of f/ ± . 

4 Suppose U is the subspace of R 4 defined by 

U = span((l, 2, 3, —4), (—5, 4, 3, 2)). 

Find an orthonormal basis of U and an orthonormal basis of II ^ . 

5 Suppose V is finite-dimensional and U is a subspace of V. Show that 
P(j± — 1 — Pu, where I is the identity operator on V. 

6 Suppose U and W are finite-dimensional subspaces of V. Prove that 
PuPw — 0 if ar *d only if (u,w) = 0 for all u e U and all w € W. 

7 Suppose V is finite-dimensional and P e C(V) is such that P 2 = P and 
every vector in null P is orthogonal to every vector in range P . Prove 
that there exists a subspace U of V such that P — Pu- 

8 Suppose V is finite-dimensional and P e C(V) is such that P 2 — P 
and 

||Pv|| < || v|| 

for every v e V. Prove that there exists a subspace U of V such that 

P = Pu- 

9 Suppose T e C(V) and U is a finite-dimensional subspace of V. Prove 
that U is invariant under T if and only if F\j T Pjj — T Pjj . 
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10 Suppose V is finite-dimensional, T s C(V), and U is a subspace 
of V. Prove that U and U 1 - are both invariant under T if and only 
if PuT = TPjj. 

11 In R 4 , let 

U = span((l, 1,0,0), (1, 1, 1,2)). 

Find u € U such that \\u — (1, 2, 3, 4)|| is as small as possible. 

12 Find p e 7^3 (R) such that p(0) = 0, p'(0) = 0, and 



— p(x) \ 2 dx 


is as small as possible. 


13 Find p € Vs (R) that makes 



sin.r — p(x)\ 2 dx 


as small as possible. 

[The polynomial 6.60 is an excellent approximation to the answer to this 
exercise, but here you are asked to find the exact solution, which involves 
powers ofn. A computer that can perform symbolic integration will be 
useful.] 


14 Suppose Cr ( [— 1 , 1]) is the vector space of continuous real-valued func- 
tions on the interval [—1,1] with inner product given by 

(fg) = J i f(x)g(x) dx 

for figs Cr([— 1, 1]). Let U be the subspace of Cr([— 1, 1]) defined 
by 

U — {f s C R ([— 1, 1]) : / (0) = 0}. 

(a) Show that U 1 - = {0}. 

(b) Show that 6.47 and 6.5 1 do not hold without the finite-dimensional 
hypothesis. 




Isaac Newton 
C 1642-1727 ), as 
envisioned by British 
poet and artist William 
Blake in this 1 795 
painting. 


Operators on Inner Product 
Spaces 


The deepest results related to inner product spaces deal with the subject 
to which we now turn — operators on inner product spaces. By exploiting 
properties of the adjoint, we will develop a detailed description of several 
important classes of operators on inner product spaces. 

A new assumption for this chapter is listed in the second bullet point below: 

7.1 Notation F, V 

• F denotes R or C. 

• V and W denote finite-dimensional inner product spaces over F. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ adjoint 

■ Spectral Theorem 

■ positive operators 

■ isometries 

■ Polar Decomposition 

■ Singular Value Decomposition 
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Self-Adjoint and Normal Operators 

Adjoints 

7.2 Definition adjoint, T* 

Suppose T € C(V , W). The adjoint of T is the function T* : W — >• V 
such that 

(Tv, w) = (v, T*w) 
for every v e V and every w e W. 


7.A 


To see why the definition above 
makes sense, suppose T e C(V, W). 
Fix w € W. Consider the linear func- 
tional on V that maps v 6 V to (Tv, w)\ 
this linear functional depends on T and 
w. By the Riesz Representation Theo- 
rem (6.42), there exists a unique vector 
in V such that this linear functional is 
given by taking the inner product with it. We call this unique vector T*w. In 
other words, T*w is the unique vector in V such that (Tv,w) — (v, T*w) for 
every v e V. 


The word adjoint has another 
meaning in linear algebra. We do 
not need the second meaning in 
this book. In case you encounter 
the second meaning for adjoint 
elsewhere, be warned that the two 
meanings for adjoint are unrelated 
to each other. 


7.3 Example Define T : R 3 — >■ R 2 by 


T(x i,x 2 ,x 3 ) = (x 2 + 3x 3 ,2xi). 

Find a formula for T*. 

Solution Flere T* will be a function from R 2 to R 3 . To compute T*, fix a 
point (yi, y 2 ) £ R 2 - Then for every (xi, x 2 , x 3 ) e R 3 we have 

((xi, x 2 , x 3 ), T*(yi, y 2 )) = (T(x\, x 2 , x 3 ), (yi, y 2 )) 

= ((x 2 + 3 x 3 ,2xi), (yi,y 2 )) 

= x 2 yi + 3x 3 yi + 2xiy 2 
= ((xi,x 2 ,x 3 ), (2y 2 ,yi,3yi)). 


Thus 


T*(yi,y 2 ) = (2y 2 ,yi,3yi). 
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7.4 Example Fix u e V and x € W. Define T e C(V, W) by 

Tv = (v, u)x 

for every v € V. Find a formula for T*. 


Solution Fix w € W. Then for every v e V we have 


Thus 


(v, T*w) = (Tv, w) 

= ((v, u)x,w ) 

— (v, u)(x, w) 

— (v, (w, x)u ). 


T*w = (w, x)u. 


In the two examples above, T* turned out to be not just a function but a 
linear map. This is true in general, as shown by the next result. 

The proofs of the next two results use a common technique: flip T* from 
one side of an inner product to become T on the other side. 


7.5 The adjoint is a linear map 
If T e C(V, W ), then T* e C(W, V ). 


Proof Suppose T e C(V , W). Fix w i , vt ’2 € W. If v € V, then 

(v, T*(w i + W 2 )) = (T v, w\ + W 2 ) 

= (Tv,wi) + ( Tv,w 2 ) 

= (v,T*wi) + (v,T*w 2 ) 

= (v,T*w 1 + T*w 2 ), 

which shows that T*(wi + W 2 ) = T*w\ + T*w 2 . 

Fix w € W and A e F. If v e F, then 

(v, T*(Xw)) = (Tv,Xw) 

— A (Tv, w) 

= A(v, r*w) 

= <v, XT*w), 

which shows that T* (Aw) = AT* tv. 

Thus 7* is a linear map, as desired. ■ 
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7.6 

Properties of the adjoint 


(a) 

(S + T r 

■ = S* + T* for all S, T e C(V, 

wy, 

(b) 

(XT)* = 

XT* for all A e F and T e C(V, 

wy 

(c) 

( T * ) * 

T for all T e C(V,W)\ 


(d) 

I * = /, where I is the identity operator on 

■ V ; 

(e) 

(ST)* = 

T*S* for all T € C(V, W) and 

5 e C(W, U) (here U 


is an inner product space over F). 


Proof 





(a) Suppose S,T e C(V, W). If v e V and w e W, then 

(v,(S + T)*w) = ((S + 7>,w) 

= (Sv, w) + (Tv, w) 

= (v, S*w) + (v, T*w) 

= (v, S*w + T*w ). 

Thus ( S + T)*w — 5*w + T*w, as desired. 

(b) Suppose A e F and T e £(K, W). If v € V and w e W, then 

(v, (A T)*w) — (XT v, w) — A (Tv, w) — A(v, T*w) — (v, A T*w). 
Thus (XT)*w — XT*w, as desired. 

(c) Suppose T e C(V, W). If v e V and w € W, then 

(w,(T*)*v) — (T*w,v) — (v,T*w) — ( Tv,w ) = (w, Tv). 
Thus ( T*)*v = Tv, as desired. 

(d) If v, u e V, then 

(v,I*u) — (Iv,u) — ( v,u ). 

Thus I*u = u, as desired. 

(e) Suppose T e C(V, W) and S € C(W, U). If v € V and u e U, then 

(v,(ST)*u) = ( STv.u ) = (Tv, S*u) = (v,T*(S*u)). 
Thus ( ST)*u — T*(S*u), as desired. 
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The next result shows the relationship between the null space and the range 
of a linear map and its adjoint. The symbol <+=+ used in the proof means “if 
and only if”; this symbol could also be read to mean “is equivalent to”. 

7.7 Null space and range of T* 

Suppose T € £( V, W). Then 

(a) null T* = (range T)^~\ 

(b) rangeT* = (null T) ± ; 

(c) null T = (rangeT*)- 1 -; 

(d) ranger = (null T *)-*-. 

Proof We begin by proving (a). Let w e W. Then 

w e null r* +=+ T*w = 0 

+=+ ( v , r*w) = 0 for all v e V 
{ Tv,w ) =0 for all v € V 
w € (range T 

Thus null r* = (range T)^, proving (a). 

If we take the orthogonal complement of both sides of (a), we get (d), 
where we have used 6.51. Replacing T with T* in (a) gives (c), where we 
have used 7.6(c). Finally, replacing T with T* in (d) gives (b). ■ 

7.8 Definition conjugate transpose 

The conjugate transpose of an m-by-n matrix is the n-by-m matrix ob- 
tained by interchanging the rows and columns and then taking the complex 
conjugate of each entry. 


7.9 Example 

The conjugate transpose of the matrix 
2 3 + 4/7 

is the matrix 



If F = R, then the conjugate trans- 
pose of a matrix is the same as its 
transpose, which is the matrix ob- 
tained by interchanging the rows 
and columns. 
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The next result shows how to com- 
pute the matrix of T* from the matrix 
of T. 

Caution: Remember that the result 
below applies only when we are dealing 
with orthonormal bases. With respect to 
nonorthonormal bases, the matrix of T* 
does not necessarily equal the conjugate 
transpose of the matrix of T. 

7.10 The matrix of T* 

Let T e C( V. W). Suppose e \, . . . , e n is an orthonormal basis of V and 
fi , • • • , fm is an orthonormal basis of W. Then 

M(T*,(f 1 ,...,f m ),(e 1 ,...,e n )) 


The adjoint of a linear map does 
not depend on a choice of basis. 
This explains why this book em- 
phasizes adjoints of linear maps 
instead of conjugate transposes of 
matrices. 


is the conjugate transpose of 


Proof In this proof, we will write M.(T) instead of the longer expres- 
sion M{T,(e i e n ), (/i fm))', we will also write M{T*) instead 

of M(T*,{f l ,...,fm),(eu...,e n j). 

Recall that we obtain the k th column of M.{T) by writing Teg as a linear 
combination of the /y’s; the scalars used in this linear combination then 
become the k th column of A4 (T). Because /]...., f m is an orthonormal 
basis of W, we know how to write Teg as a linear combination of the /y’s 
(see 6.30): 

Te k = {Teg, /i)/i + ■■■ + {Te k ,f m )f m . 

Thus the entry in row j , column k, of A i(T) is {Teg, fj). 

Replacing T with T* and interchanging the roles played by the <?’s and 
/’ s, we see that the entry in row j , column k, of M(T*) is {T* fg,ej), 
which equals { fg,Tej ), which equals {Tej, fg), which equals the complex 
conjugate of the entry in row k, column j , of M. {T). In other words, M.{T*) 
is the conjugate transpose of M.( T). ■ 
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Self-Adjoint Operators 

Now we switch our attention to operators on inner product spaces. Thus 
instead of considering linear maps from V to W, we will be focusing on linear 
maps from V to V; recall that such linear maps are called operators. 

7.11 Definition self-adjoint 

An operator T e C(V) is called self-adjoint if T = T*. In other words, 

T e C(V) is self-adjoint if and only if 

(Tv, w) = (v, T w ) 

for all v,w € V. 


7.12 Example Suppose T is the operator on F 2 whose matrix (with re- 
spect to the standard basis) is 


2 b 

3 7 

Find all numbers b such that T is self-adjoint. 

Solution The operator T is self-adjoint if and only if b — 3 (because 
M.(T) = A4 (T*j if and only i f h — 3 ; recall that M.(T*) is the conjugate 
transpose of M(T ) — see 7.10). 


You should verify that the sum of two self-adjoint operators is self-adjoint 
and that the product of a real scalar and a self-adjoint operator is self-adjoint. 

A good analogy to keep in mind (es- 
pecially when F = C) is that the adjoint 
on C( V) plays a role similar to complex 
conjugation on C. A complex number 
z is real if and only if z = z; thus a self- 
adjoint operator (T = T*) is analogous 
to a real number. 

We will see that the analogy discussed above is reflected in some important 
properties of self-adjoint operators, beginning with eigenvalues in the next 
result. 

If F = R, then by definition every eigenvalue is real, so the next result is 
interesting only when F = C. 


Some mathematicians use the term 
Hermitian instead of self-adjoint, 
honoring French mathematician 
Charles Hermite, who in 1873 pub- 
lished the first proof that e is not a 
zero of any polynomial with integer 
coefficients. 
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7.13 Eigenvalues of self-adjoint operators are real 
Every eigenvalue of a self-adjoint operator is real. 

Proof Suppose T is a self-adjoint operator on V. Let A be an eigenvalue of 
T, and let v be a nonzero vector in V such that Tv — Av. Then 

A || i’|| 2 = (Av, v) = (Tv,v) = (v. Tv) = (v, Av) = A||v|| 2 . 

Thus A = A, which means that A is real, as desired. ■ 

The next result is false for real inner product spaces. As an example, 
consider the operator T e £(R 2 ) that is a counterclockwise rotation of 90° 
around the origin; thus T(x,y) = (— y, x ). Obviously T v is orthogonal to v 
for every v € R 2 , even though T ^ 0. 

7.14 Over C, Tv is orthogonal to v for all v only for the 0 operator 
Suppose V is a complex inner product space and T e C(V). Suppose 

(T v, v) = 0 


for all v e V. Then T = 0. 


Proof We have 


(Tu,w) — 


(T(u + w ), u + w) — (T(u — w), u —w) 


(T{u + i w ) , u + iw) — (T{u — iw ), u — iw) . 
I ~ r 


for all u , w e V, as can be verified by computing the right side. Note that 
each term on the right side is of the form (Tv, v) for appropriate v e V. Thus 
our hypothesis implies that ( Tu,w ) = 0 for all u,w e V. This implies that 
T — 0 (take w — T u). ■ 


The next result provides another ex- 
ample of how self-adjoint opera- 
tors behave like real numbers. 


The next result is false for real inner 
product spaces, as shown by consider- 
ing any operator on a real inner product 
space that is not self-adjoint. 
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7.15 Over C, (Tv, v) is real for all v only for self-adjoint operators 

Suppose V is a complex inner product space and T € C(V). Then T is 
self-adjoint if and only if 

(Tv, v) e R 


for every v e V. 


Proof Let v e V. Then 

(Tv, v) — (Tv, v) = (Tv, v) — (v, Tv) = (Tv,v) - (T*v,v) = ((T-T*)v,v). 

If (Tv,v) e R for every v € V, then the left side of the equation above equals 
0, so ((T — T*)v, v) = 0 for every v e V. This implies that T — T* = 0 (by 
7. 14). Hence T is self-adjoint. 

Conversely, if T is self-adjoint, then the right side of the equation above 
equals 0, so (Tv, v) — (Tv, v) for every v € V. This implies that (Tv, v) € R 
for every v e V, as desired. ■ 

On a real inner product space V, a nonzero operator T might satisfy 
(Tv,v) — 0 for all v e V. However, the next result shows that this cannot 
happen for a self-adjoint operator. 


7.16 If T — T* and (T v, v) = 0 for all v, then T = 0 
Suppose T is a self-adjoint operator on V such that 

(7V, v) = 0 


for all v € V. Then T — 0. 

Proof We have already proved this (without the hypothesis that T is self- 
adjoint) when V is a complex inner product space (see 7.14). Thus we can 
assume that V is a real inner product space. If u, w e V, then 

(T(u + w),u + w) - (T(u -w),u -w) 

7.17 ( Tu,w ) — ; 

x 1 4 

this is proved by computing the right side using the equation 
( Tw,u ) — ( w,Tu ) = (Tu,w), 

where the first equality holds because T is self-adjoint and the second equality 
holds because we are working in a real inner product space. 

Each term on the right side of 7.17 is of the form (Tv, v) for appropriate v. 
Thus (Tu,w) =0 for all u,w e V. This implies that T — 0 (take w = Tu). m 
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Normal Operators 
7.18 Definition normal 

• An operator on an inner product space is called normal if it com- 
mutes with its adjoint. 

• In other words, T e C(V) is normal if 


Obviously every self-adjoint operator is normal, because if T is self-adjoint 
then T* = T. 


7.19 Example Let T be the operator on F 2 whose matrix (with respect to 
the standard basis) is 



Show that T is not self-adjoint and that T is normal. 

Solution This operator is not self-adjoint because the entry in row 2, column 1 
(which equals 3) does not equal the complex conjugate of the entry in row 1, 
column 2 (which equals —3). 

The matrix of TT* equals 

0 ~2 ) ( — 3 2)'" et l“ ls ( 0 13)' 

Similarly, the matrix of T*T equals 

( —3 2 ) ( 3 ^ )' WhicheqUa ‘ S ( 0 13 )' 

Because TT* and T*T have the same matrix, we see that TT* = T*T. 
Thus T is normal. 


The next result implies that 
null T = null T* for every normal 
operator T. 


In the next section we will see why 
normal operators are worthy of special 
attention. 

The next result provides a simple 
characterization of normal operators. 
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7.20 T is normal if and only if ||7v|| = ||T*v|| for all v 
An operator T e C{V) is normal if and only if 

II t v|| = ||r*v|| 


for all v € V. 

will prove both directions of this result at the same 

rj-i sf: rj-i rj-i rj~ l >(c Q 

{(T*T- TT*)v,v) = 0 for all v eV 
{T*Tv,v) = {TT*v,v) for all v e V 
|| T v 1 1 2 = || r*v|| 2 for all v € V, 

where we used 7.16 to establish the second equivalence (note that the operator 
T*T - TT* is self-adjoint). The equivalence of the first and last conditions 
above gives the desired result. ■ 

Compare the next corollary to Exercise 2. That exercise states that the 
eigenvalues of the adjoint of each operator are equal (as a set) to the complex 
conjugates of the eigenvalues of the operator. The exercise says nothing 
about eigenvectors, because an operator and its adjoint may have different 
eigenvectors. However, the next corollary implies that a normal operator and 
its adjoint have the same eigenvectors. 

7.21 For T normal, T and T* have the same eigenvectors 

Suppose T e C(V) is normal and v € V is an eigenvector of T with 
eigenvalue A. Then v is also an eigenvector of T* with eigenvalue A. 

Proof Because T is normal, so is T — XI, as you should verify. Using 7.20, 
we have 


Proof Let T € C{V). We 
time. Note that 


T is normal 


0 = || (T - A/ )v || = || (T- A/)*v|| = || (T* - A/)v|| . 

Hence v is an eigenvector of T* with eigenvalue A, as desired. ■ 

Because every self-adjoint operator is normal, the next result applies in 
particular to self-adjoint operators. 
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7.22 Orthogonal eigenvectors for normal operators 

Suppose T e £{ V ) is normal. Then eigenvectors of T corresponding to 
distinct eigenvalues are orthogonal. 

Proof Suppose a, ft are distinct eigenvalues of T, with corresponding eigen- 
vectors u, v. Thus Tu — au and Tv = fiv. From 7.21 we have T*v = pv. 
Thus 


(a — P){u, v) = (aw, v) — (u, fiv) 

= (T u,v) — { u , T*v) 

= 0. 

Because a ^ ft, the equation above implies that (u, v) — 0. Thus u and v are 
orthogonal, as desired. ■ 


EXERCISES 7 . A 


1 Suppose n is a positive integer. Define T e £( F”) by 

T(z\ z„) = (0, zi, . . . ,z n _i). 

Find a formula for T*(z \, . . . , z n ). 

2 Suppose T € C(V) and A e F. Prove that A is an eigenvalue of T if and 
only if A is an eigenvalue of T*. 

3 Suppose T e £( V) and U is a subspace of V. Prove that U is invariant 
under T if and only if (A is invariant under T*. 

4 Suppose T € C{V, W). Prove that 

(a) T is injective if and only if T* is surjective; 

(b) T is surjective if and only if T* is injective. 

5 Prove that 

dim null T* = dim null T + dim W — dim V 

and 

dim range T* = dim range T 
for every T e £( V. W). 
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6 Make T *2 (R) into an inner product space by defining 

( P,q) = / p(x)q(x)dx. 

Jo 

Define T e /^(T^IR)) by T(ao + a\x + a^x 2 ) = a\x. 

(a) Show that T is not self-adjoint. 

(b) The matrix of T with respect to the basis (1, x, x 2 ) is 



This matrix equals its conjugate transpose, even though T is not 
self-adjoint. Explain why this is not a contradiction. 


7 Suppose S,T e C(V) are self-adjoint. Prove that NT is self-adjoint if 
and only if ST = TS. 

8 Suppose V is a real inner product space. Show that the set of self-adjoint 
operators on V is a subspace of C(V). 

9 Suppose V is a complex inner product space with V ^ {0}. Show that 
the set of self-adjoint operators on V is not a subspace of C(V). 

10 Suppose dim V >2. Show that the set of normal operators on V is not a 
subspace of C(V). 

11 Suppose P € C(V) is such that P 2 — P . Prove that there is a subspace 
U of V such that P — Pu if and only if P is self-adjoint. 

12 Suppose that T is a normal operator on V and that 3 and 4 are eigenvalues 
of T. Prove that there exists a vector v e V such that ||v|| = a/ 2 and 

IM = 5. 

13 Give an example of an operator T € £(C 4 ) such that T is normal but 
not self-adjoint. 

14 Suppose T is a normal operator on V. Suppose also that v, w € V satisfy 
the equations 


i>|| = || w || =2, Tv — 3v, Tw — Aw. 


Show that || T(v + w) || = 10. 
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15 Fix u, x e V. Define T € C{V) by 


Tv — (v, u)x 


for every v e V. 

(a) Suppose F = R. Prove that T is self-adjoint if and only if u, x is 
linearly dependent. 

(b) Prove that T is normal if and only if u, x is linearly dependent. 
16 Suppose T € C{V) is normal. Prove that 


range T = range T* . 


17 Suppose T e C(V) is normal. Prove that 

null T k — null T and range T k = range T 


for every positive integer k. 

18 Prove or give a counterexample: If T s C(V) and there exists an ortho- 
normal basis ei,...,e n of V such that || T ej || = || T*ej || for each j , 
then T is normal. 

19 Suppose T e £( C 3 ) is normal and T{\, 1, 1) = (2,2,2). Suppose 
(zi, Z2, Z 3 ) e null T. Prove that zi + Z 2 + Z 3 — 0. 

20 Suppose T € £(F, W) and F = R. Let $>v be the isomorphism from V 
onto the dual space V' given by Exercise 17 in Section 6.B, and let 

be the corresponding isomorphism from W onto W’. Show that if <&y and 
are used to identify Land W with V' and W\ then T* is identified 
with the dual map T'. More precisely, show that O7 o T* = T' 0%. 

21 Fix a positive integer n. In the inner product space of continuous real- 
valued functions on [— n, Jt] with inner product 



let 


V = span(l, cos x, cos 2x , . . . , cos nx, sinx, sin2x, . . . , sinnx). 

(a) Define D e C(V) by Df — f. Show that D* = —D. Conclude 
that D is normal but not self-adjoint. 

(b) Define T e C(V) by Tf = f". Show that T is self-adjoint. 
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7.B The Spectral Theorem 


Recall that a diagonal matrix is a square matrix that is 0 everywhere except 
possibly along the diagonal. Recall also that an operator on V has a diagonal 
matrix with respect to a basis if and only if the basis consists of eigenvectors 
of the operator (see 5.41). 

The nicest operators on V are those for which there is an orthonormal 
basis of V with respect to which the operator has a diagonal matrix. These 
are precisely the operators T e C( V) such that there is an orthonormal basis 
of V consisting of eigenvectors of T. Our goal in this section is to prove the 
Spectral Theorem, which characterizes these operators as the normal operators 
when F = C and as the self-adjoint operators when F = R. The Spectral 
Theorem is probably the most useful tool in the study of operators on inner 
product spaces. 

Because the conclusion of the Spectral Theorem depends on F, we will 
break the Spectral Theorem into two pieces, called the Complex Spectral 
Theorem and the Real Spectral Theorem. As is often the case in linear algebra, 
complex vector spaces are easier to deal with than real vector spaces. Thus 
we present the Complex Spectral Theorem first. 

The Complex Spectral Theorem 

The key part of the Complex Spectral Theorem (7.24) states that if F = C 
and T e C{ V) is normal, then T has a diagonal matrix with respect to some 
orthonormal basis of V. The next example illustrates this conclusion. 

7.23 Example Consider the normal operator T e £(C 2 ) from Example 
7. 19, whose matrix (with respect to the standard basis) is 




eigenvectors of T, and with respect to this basis the matrix of T is the diagonal 
matrix 


( 


2 + 3 i 0 
0 2-3 i 


In the next result, the equivalence of (b) and (c) is easy (see 5.41). Thus 
we prove only that (c) implies (a) and that (a) implies (c). 
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7.24 Complex Spectral Theorem 

Suppose F = C and T e £(F). Then the following are equivalent: 

(a) T is normal. 

(b) V has an orthonormal basis consisting of eigenvectors of T. 

(c) T has a diagonal matrix with respect to some orthonormal basis 
of V. 


Proof First suppose (c) holds, so T has a diagonal matrix with respect to 
some orthonormal basis of V. The matrix of T* (with respect to the same 
basis) is obtained by taking the conjugate transpose of the matrix of T ; hence 
T* also has a diagonal matrix. Any two diagonal matrices commute; thus T 
commutes with T*, which means that T is normal. In other words, (a) holds. 

Now suppose (a) holds, so T is normal. By Schur’s Theorem (6.38), 
there is an orthonormal basis e\, ... ,e n of V with respect to which T has an 
upper-triangular matrix. Thus we can write 


7.25 




( <71,1 

V 0 


<7l,« ^ 
®n,n / 


We will show that this matrix is actually a diagonal matrix. 

We see from the matrix above that 

||7>i|| 2 = |ai,i| 2 

and 

|| T e\\\ — |$i,i| + |^2 1 , 2 1 + • • • + |tfi,H | • 

Because T is normal, || Te \ || = || T* e \ || (see 7.20). Thus the two equations 
above imply that all entries in the first row of the matrix in 7.25, except 
possibly the first entry a\ } \, equal 0. 

Now from 7.25 we see that 

\\Te 2 \\ 2 — | ^2 2 ,2 1 2 

(because a\^ 2 — 0, as we showed in the paragraph above) and 
\\T*e 2 \\ 2 = \ci 2 ,2 1 ~ + |<72,3 1 - + • • • + \a 2 ,n\~ ■ 

Because T is normal, || Te 2 \\ — || T*e 2 1| . Thus the two equations above imply 
that all entries in the second row of the matrix in 7.25, except possibly the 
diagonal entry a 2 , 2 , equal 0. 

Continuing in this fashion, we see that all the nondiagonal entries in the 
matrix 7.25 equal 0. Thus (c) holds. ■ 
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The Real Spectral Theorem 


We will need a few preliminary results, which apply to both real and complex 
inner product spaces, for our proof of the Real Spectral Theorem. 

You could guess that the next result 
is true and even discover its proof by 
thinking about quadratic polynomials 
with real coefficients. Specifically, sup- 
pose /),ceR and b 2 < 4c. Let x be a 
real number. Then 


This technique of completing the 
square can be used to derive the 
quadratic formula. 


x 2 + bx + c 



> 0 . 


In particular, x 2 + bx + c is an invertible real number (a convoluted way 
of saying that it is not 0). Replacing the real number x with a self-adjoint 
operator (recall the analogy between real numbers and self-adjoint operators), 
we are led to the result below. 


7.26 Invertible quadratic expressions 

Suppose T € C(V) is self-adjoint and b, c € R are such that b 2 < 4c. 
Then 

T 2 + bT + cl 


is invertible. 


Proof Let v be a nonzero vector in V. Then 

({T 2 + bT + c/)v, v) = (T 2 v, v) + b{Tv, v) + c(v, v) 

= (Tv, Tv) + b(Tv,v) + c || v|| 2 
> ||Tv|| 2 - |6|||rv||||v|| + c||v|| 2 
|&|||v||\ 2 / b 2 ' 


= (\\Tv\\ 
> 0 




where the third line above holds by the Cauchy-Schwarz Inequality (6.15). 
The last inequality implies that (T 2 + bT + cl)v 7^ 0. Thus T 2 + bT + cl 
is injective, which implies that it is invertible (see 3.69). ■ 


We know that every operator, self-adjoint or not, on a finite-dimensional 
nonzero complex vector space has an eigenvalue (see 5.21). Thus the next 
result tells us something new only for real inner product spaces. 
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7.27 Self-adjoint operators have eigenvalues 

Suppose V 7^ {0} and T e C(V) is a self-adjoint operator. Then T has 
an eigenvalue. 

Proof We can assume that V is a real inner product space, as we have already 
noted. Let n — dim V and choose v e V with v ^ 0. Then 

v, Tv, T 2 v, ...,T n v 

cannot be linearly independent, because V has dimension n and we have n + 1 
vectors. Thus there exist real numbers ao a n , not all 0, such that 

0 = aov + a i Tv + • • ■ + a n T n v. 

Make the a ’s the coefficients of a polynomial, which can be written in factored 
form (see 4.17) as 


ao + a\x + • ■ ■ + a n x n 

— c(x 2 + bix + ci) ■ ■ ■ (x 2 + b M x + cm)(x - Aj.) • • • (x - X m ), 

where c is a nonzero real number, each bj, Cj, and A j is real, each h j 2 is less 
than 4cy , m + M > 1, and the equation holds for all real x. We then have 

0 = aov + a\T v + ■■■ + a n T n v 
= (ciol + a.\T + • • • + a n T n )v 

= c(T 2 + b 1 T + c 1 I)---(T 2 + b M T + c M I)(T-X 1 I)---(T- A m I)v. 

By 7.26, each T 2 + bjT + cjl is invertible. Recall also that c / 0. Thus 
the equation above implies that m > 0 and 


0 = {T — X \ I) ■■■ (T — X m I)v. 


Hence T — Xjl is not injective for at least one j . In other words, T has an 
eigenvalue. ■ 

The next result shows that if U is a subspace of V that is invariant under 
a self-adjoint operator T, then U 1 - is also invariant under T. Later we will 
show that the hypothesis that T is self-adjoint can be replaced with the weaker 
hypothesis that T is normal (see 9.30). 
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7.28 Self-adjoint operators and invariant subspaces 

Suppose T e C(V) is self-adjoint and U is a subspace of V that is 
invariant under T. Then 

(a) U 1 - is invariant under T ; 

(b) T\u e C(U) is self-adjoint; 

(c) T\u± € £(£/-*-) is self-adjoint. 

Proof To prove (a), suppose v e U ■*■. Let u € U. Then 

(Tv, u) = (v, T u) = 0, 

where the first equality above holds because T is self-adjoint and the second 
equality above holds because U is invariant under T (and hence Tu € U) 
and because v € U Because the equation above holds for each u e U, we 
conclude that Tv € U ~ L . Thus is invariant under T, completing the proof 
of (a). 

To prove (b), note that if u, v € U, then 

((T\u)u,v) = (Tu, v) = (u, T v) = (u,(T\ v )v). 

Thus T\u is self-adjoint. 

Now (c) follows from replacing U with U 1 - in (b), which makes sense 
by (a). ■ 

We can now prove the next result, which is one of the major theorems in 
linear algebra. 

7.29 Real Spectral Theorem 

Suppose F = R and T e C(V). Then the following are equivalent: 

(a) T is self-adjoint. 

(b) V has an orthonormal basis consisting of eigenvectors of T. 

(c) T has a diagonal matrix with respect to some orthonormal basis 
of V. 
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Proof First suppose (c) holds, so T has a diagonal matrix with respect to 
some orthonormal basis of V. A diagonal matrix equals its transpose. Hence 
T — T*, and thus T is self-adjoint. In other words, (a) holds. 

We will prove that (a) implies (b) by induction on dim V. To get started, 
note that if dim V = 1, then (a) implies (b). Now assume that dim V > 1 and 
that (a) implies (b) for all real inner product spaces of smaller dimension. 

Suppose (a) holds, so T e C(V) is self-adjoint. Let u be an eigenvector 
of T with ||m|| = 1 {1 .21 guarantees that T has an eigenvector, which can 
then be divided by its norm to produce an eigenvector with norm 1). Let 
U — span(w). Then U is a 1 -dimensional subspace of V that is invariant 
under T. By 7.28(c), the operator T\jj± e C(U^) is self-adjoint. 

By our induction hypothesis, there is an orthonormal basis of U 1 - consist- 
ing of eigenvectors of T\u±. Adjoining u to this orthonormal basis of U 1 - 
gives an orthonormal basis of V consisting of eigenvectors of T. completing 
the proof that (a) implies (b). 

We have proved that (c) implies (a) and that (a) implies (b). Clearly (b) 
implies (c), completing the proof. ■ 


7.30 Example Consider the self-adjoint operator T on R 3 whose matrix 
(with respect to the standard basis) is 



As you can verify, 


( 1 .- 1 . 0 ) ( 1 - 1 - 1 ) ( 1 , 1 .- 2 ) 

Vi ’ V3 ’ V6 

is an orthonormal basis of R 3 consisting of eigenvectors of T, and with respect 
to this basis, the matrix of T is the diagonal matrix 

1 0 0 




0 9 0 

0 0 -15 


If F = C, then the Complex Spectral Theorem gives a complete descrip- 
tion of the normal operators on V. A complete description of the self-adjoint 
operators on V then easily follows (they are the normal operators on V whose 
eigenvalues all are real; see Exercise 6). 

If F = R, then the Real Spectral Theorem gives a complete description 
of the self-adjoint operators on V. In Chapter 9, we will give a complete 
description of the normal operators on V (see 9.34). 
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EXERCISES 7.B 


1 True or false (and give a proof of your answer): There exists T e £(R 3 ) 
such that T is not self-adjoint (with respect to the usual inner product) 
and such that there is a basis of R 3 consisting of eigenvectors of T. 

2 Suppose that T is a self-adjoint operator on a finite-dimensional inner 
product space and that 2 and 3 are the only eigenvalues of T. Prove that 
T 2 -5T + 61 = 0. 

3 Give an example of an operator T e £(C 3 ) such that 2 and 3 are the 
only eigenvalues of T and T 2 — 5T + 61 ^ 0. 

4 Suppose F = C and T e C{ V). Prove that T is normal if and only if 
all pairs of eigenvectors corresponding to distinct eigenvalues of T are 
orthogonal and 

V = E{X u T)®---®E(X m ,T), 
where Ai , . . . , X m denote the distinct eigenvalues of T. 

5 Suppose F = R and T e C(V). Prove that T is self-adjoint if and only 
if all pairs of eigenvectors corresponding to distinct eigenvalues of T are 
orthogonal and 

V = E(X 1 ,T)@~-®E(X m ,T), 
where X i , . . . , X m denote the distinct eigenvalues of T. 

6 Prove that a normal operator on a complex inner product space is self- 
adjoint if and only if all its eigenvalues are real. 

[The exercise above strengthens the analogy (for normal operators) 
between self-adjoint operators and real numbers .] 

7 Suppose V is a complex inner product space and T € C(V) is a normal 
operator such that T 9 = T 8 . Prove that T is self-adjoint and T 2 = T. 

8 Give an example of an operator T on a complex vector space such that 
T 9 = T 8 but T 2 ^ T. 

9 Suppose V is a complex inner product space. Prove that every normal 
operator on V has a square root. (An operator S € C( V) is called a 
square root of T e C(V) if S 2 — T.) 
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10 Give an example of a real inner product space V and T € C( V) and real 
numbers b , c with b 2 < 4c such that T 2 + bT + cl is not invertible. 
[The exercise above shows that the hypothesis that T is self-adjoint is 
needed in 7.26, even for real vector spaces .] 

11 Prove or give a counterexample: every self-adjoint operator on V has a 
cube root. (An operator S € C(V) is called a cube root of T € C(V) if 
S 3 = T.) 

12 Suppose T s C( V) is self-adjoint, A e F, and e > 0. Suppose there 
exists v € V such that ||v|| = 1 and 

|| Tv — Av || < e. 

Prove that T has an eigenvalue A' such that |A — A'| < e. 

13 Give an alternative proof of the Complex Spectral Theorem that avoids 
Schur’s Theorem and instead follows the pattern of the proof of the Real 
Spectral Theorem. 

14 Suppose U is a finite-dimensional real vector space and T e C(U). 
Prove that U has a basis consisting of eigenvectors of T if and only if 
there is an inner product on U that makes T into a self-adjoint operator. 

15 Find the matrix entry below that is covered up. 



keep away from him., 
he's not norma 1 1 


112 
1 1 1 
1 1 1 
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Positive Operators and Isometries 

Positive Operators 

7.31 Definition positive operator 

An operator T e C{V) is called positive if T is self-adjoint and 

(Tv, v> > 0 

for all v € V. 

If V is a complex vector space, then the requirement that T is self-adjoint 
can be dropped from the definition above (by 7 . 15 ). 

7.32 Example positive operators 

(a) If U is a subspace of V, then the orthogonal projection Pjj is a positive 
operator, as you should verify. 

(b) If T € C(V) is self-adjoint and b, c € R are such that b * 2 < 4 c, then 
T 2 + bT + cl is a positive operator, as shown by the proof of 7 . 26 . 


7.C 


7.33 Definition square root 

An operator R is called a square root of an operator T if R 2 = T. 


7.34 Example If T e £(F 3 ) is defined by T(z\, 22,23) = (23, 0 , 0 ), 
then the operator R e £(F 3 ) defined by R(z \ , 22, "3) = (22,23,0) is a 
square root of T. 


The characterizations of the positive 
operators in the next result correspond 
to characterizations of the nonnegative 
numbers among C. Specifically, a com- 
plex number 2 is nonnegative if and 
only if it has a nonnegative square root, 
corresponding to condition (c). Also, 

2 is nonnegative if and only if it has a real square root, corresponding to 

condition (d). Finally, 2 is nonnegative if and only if there exists a complex 
number w such that 2 = vvw, corresponding to condition (e). 


The positive operators correspond 
to the numbers [0,oo), so better 
terminology would use the term 
nonnegative instead of positive. 
However, operator theorists consis- 
tently call these the positive opera- 
tors, so we will follow that custom. 
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7.35 Characterization of positive operators 
Let T e £(F). Then the following are equivalent: 

(a) T is positive; 

(b) T is self-adjoint and all the eigenvalues of T are nonnegative; 

(c) T has a positive square root; 

(d) T has a self-adjoint square root; 

(e) there exists an operator R e C{V) such that T = R* R. 

Proof We will prove that (a) =>■ (b) =>■ (c) =>■ (d) =>• (e) =$■ (a). 

First suppose (a) holds, so that T is positive. Obviously T is self-adjoint 
(by the definition of a positive operator). To prove the other condition in (b), 
suppose A is an eigenvalue of T. Let v be an eigenvector of T corresponding 
to A. Then 

0 < (Tv,v) = (Av, v) = A(v, v). 

Thus A is a nonnegative number. Hence (b) holds. 

Now suppose (b) holds, so that T is self-adjoint and all the eigenvalues 
of T are nonnegative. By the Spectral Theorem (7.24 and 7.29), there is 
an orthonormal basis e\, ... ,e n of V consisting of eigenvectors of T. Let 
Ai, . . . , A„ be the eigenvalues of T corresponding to e \ , . . . , e n ,\ thus each 
A j is a nonnegative number. Let R be the linear map from V to Lsuch that 



for j = (see 3.5). Then R is a positive operator, as you should verify. 

Furthermore, R 2 ej = A jej = Tej for each j , which implies that R 2 = T. 
Thus R is a positive square root of T. Hence (c) holds. 

Clearly (c) implies (d) (because, by definition, every positive operator is 
self-adjoint). 

Now suppose (d) holds, meaning that there exists a self-adjoint operator 
R on V such that T — R 2 . Then T = R* R (because R* — R). Hence (e) 
holds. 

Finally, suppose (e) holds. Let R e C(V) be such that T = R* R. Then 
T* = (R*R)* = R*(R*f = R*R = T. Hence T is self-adjoint. To 
complete the proof that (a) holds, note that 

(Tv, v) = (R*Rv,v) = (Rv, Rv) > 0 

for every v e V. Thus T is positive. ■ 
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Each nonnegative number has a 
unique nonnegative square root. The 
next result shows that positive operators 
enjoy a similar property. 


Some mathematicians also use the 
term positive semidefinite opera- 
tor, which means the same as posi- 
tive operator. 


7.36 Each positive operator has only one positive square root 
Every positive operator on V has a unique positive square root. 


Proof Suppose T e C(V) is positive. 

Suppose v € V is an eigenvector of T. 

Thus there exists A > 0 such that Tv — 

Av. 

Let R be a positive square root of T. 

We will prove that Rv = VXv. This 
will imply that the behavior of R on the eigenvectors of T is uniquely deter- 
mined. Because there is a basis of V consisting of eigenvectors of T (by the 
Spectral Theorem), this will imply that R is uniquely determined. 

To prove that Rv = VXv, note that the Spectral Theorem asserts that 
there is an orthonormal basis e\,...,e n of V consisting of eigenvectors of R. 
Because R is a positive operator, all its eigenvalues are nonnegative. Thus 

there exist nonnegative numbers Ai X n such that Re j = VXVr f° r 

• ,n. 

Because e \ , . . . , e n is a basis of V, we can write 


A positive operator can have in- 
finitely many square roots (al- 
though only one of them can be 
positive). For example, the identity 
operator on V has infinitely many 
square roots if dim V > 1 . 


V — + • ■ ■ + Cl n e n 

for some numbers a\ a n e F. Thus 

Rv = a i VXjXi + • • • + a n V A„ e n 

and hence 

R 2 v = a\X\e\ H b a n X n e n . 

But R 2 — T, and Tv — Av. Thus the equation above implies 

ci i + • • • + a n Xe n — u\X\C\ + ••• + ci n X n 6 n . 

The equation above implies that a j (A — Xj) = 0 for j = 1 , ,n. Hence 

v= ^ cue, 


1 j^j > 


{j:Xj=X} 


and thus 


Rv = ^ cij VX ej = VXv, 

[j : A j =A} 


as desired. 
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Isometries 

Operators that preserve norms are sufficiently important to deserve a name: 

7.37 Definition isometry 

• An operator S e C(V) is called an isometry if 

l|Sv|| = IM| 

for all v € V. 

• In other words, an operator is an isometry if it preserves norms. 

For example, XI is an isometry 
whenever A e F satisfies |A| = 1. We 
will see soon that if F = C, then the 
next example includes all isometries. 

7.38 Example Suppose Ai X„ are scalars with absolute value 1 and 

S e C(V) satisfies Sej = A jej for some orthonormal basis e\, ... ,e n of V. 
Show that S is an isometry. 

Solution Suppose v € V. Then 

7.39 v = (v, e\)e\ H b (v, e n )e n 

and 

7.40 ||v|| 2 = \{v,ei)\ 2 + ••• + \{v,e n )\ 2 , 

where we have used 6.30. Applying S to both sides of 7.39 gives 


The Greek word isos means equal; 
the Greek word metron means 
measure. Thus isometry literally 
means equal measure. 


Sv = (v, e\)Se\ 4 h (v,e n )Se n 

= Ai(v, e\)e\ 4 b X n (v,e n )e n . 


The last equation, along with the equation | A / 1 = 1, shows that 

7.41 ||Sv|| 2 = |(v^ l) | 2 + ... + |(v, e „ ) |2. 

Comparing 7.40 and 7.41 shows that ||v|| = || iSv|| . In other words, S is an 
isometry. 
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The next result provides several con- 
ditions that are equivalent to being an 
isometry. The equivalence of (a) and (b) 
shows that an operator is an isometry if 
and only if it preserves inner products. 

The equivalence of (a) and (c) [or (d)] 
shows that an operator is an isometry 
if and only if the list of columns of its 
matrix with respect to every [or some] basis is orthonormal. Exercise 10 
implies that in the previous sentence we can replace “columns” with “rows”. 

7.42 Characterization of isometries 

Suppose S € C( V ). Then the following are equivalent: 

(a) S is an isometry; 

(b) (Su, Sv) = (u, v) for all u, v e V; 

(c) Sei , . . . , Se n is orthonormal for every orthonormal list of vectors 
ei, . . . ,e„ in V; 

(d) there exists an orthonormal basis e\,...,e n of V such that 
Se i, . . . , Se n is orthonormal; 

(e) S*S = /; 

(f) SS* = /; 

(g) S'* is an isometry; 

(h) S is invertible and S -1 = S*. 

Proof First suppose (a) holds, so S is an isometry. Exercises 19 and 20 in 
Section 6. A show that inner products can be computed from norms. Because 
S preserves norms, this implies that S preserves inner products, and hence 
(b) holds. More precisely, if V is a real inner product space, then for every 
u , v € V we have 

{Su,Sv) = (||5m + Sv|| 2 -||Sm-Sv|| 2 )/4 
= (l|5(n + v)|| 2 -||S(n — v)|| 2 )/4 
= (||n + v|| 2 — || u — v 1 1 2 ) / 4 
= (w,v), 


An isometry on a real inner product 
space is often called an orthogonal 
operator. An isometry on a com- 
plex inner product space is often 
called a unitary operator. We use 
the term isometry so that our re- 
sults can apply to both real and 
complex inner product spaces. 
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where the first equality comes from Exercise 19 in Section 6. A, the second 
equality comes from the linearity of 5 , the third equality holds because 5 is an 
isometry, and the last equality again comes from Exercise 19 in Section 6.A. 
If V is a complex inner product space, then use Exercise 20 in Section 6. A 
instead of Exercise 19 to obtain the same conclusion. In either case, we see 
that (b) holds. 

Now suppose (b) holds, so 5 preserves inner products. Suppose that 
e \, . . . , e n is an orthonormal list of vectors in V. Then we see that the list 
Se i, . . . , Se n is orthonormal because (Sej , Se k ) = (e j , e k ). Thus (c) holds. 

Clearly (c) implies (d). 

Now suppose (d) holds. Let e \ , . . . , e n be an orthonormal basis of V such 
that Se i, . . . , Se n is orthonormal. Thus 

(S*Sej,e k ) = (ej,e k ) 

for j,k — 1, ... ,/3 [because the term on the left equals (Sej, Se k ) and 
(Sei, . . . , Se n ) is orthonormal]. All vectors u,v e V can be written as 
linear combinations of e \, . . . , e n , and thus the equation above implies that 
(S*Su,v) = (u,v). Hence S*S = /; in other words, (e) holds. 

Now suppose (e) holds, so that S* S = I. In general, an operator S need 
not commute with S*. However, S* S = I if and only if 55* = /; this is a 
special case of Exercise 10 in Section 3.D. Thus 55* = /, showing that (f) 
holds. 

Now suppose (f) holds, so 55* = /. If v e V, then 

||5*v[| 2 = (5*v, 5*v) = (55*v, v) = (v, v) = ||r’|| 2 . 

Thus 5* is an isometry, showing that (g) holds. 

Now suppose (g) holds, so 5* is an isometry. We know that (a) =>■ (e) and 
(a) =5 (f) because we have shown (a) =>■ (b) => (c) =>■ (d) =>■ (e) =5 (f). Using 
the implications (a) =>■ (e) and (a) (f) but with 5 replaced with 5* [and 

using the equation (5*)* = 5], we conclude that 55* = I and 5*5 = I. 
Thus 5 is invertible and 5~' = 5*; in other words, (h) holds. 

Now suppose (h) holds, so 5 is invertible and 5 -1 = 5*. Thus 5*5 — I. 
If v € V, then 


|| 5v|| 2 = (5v, 5v) = (5*5v, v) = (v,v) = ||r’|| 2 . 

Thus 5 is an isometry, showing that (a) holds. 

We have shown (a) => (b) => (c) =$■ (d) (e) => (f) => (g) => (h) =>■ (a), 

completing the proof. ■ 
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The previous result shows that every isometry is normal [see (a), (e), and 
(f) of 7.42]. Thus characterizations of normal operators can be used to give 
descriptions of isometries. We do this in the next result in the complex case 
and in Chapter 9 in the real case (see 9.36). 

7.43 Description of isometries when F = C 

Suppose V is a complex inner product space and S e C(V). Then the 
following are equivalent: 

(a) S is an isometry. 

(b) There is an orthonormal basis of V consisting of eigenvectors of S 
whose corresponding eigenvalues all have absolute value 1 . 

Proof We have already shown (see Example 7.38) that (b) implies (a). 

To prove the other direction, suppose (a) holds, so S is an isometry. By the 
Complex Spectral Theorem (7.24), there is an orthonormal basis e\, ... ,e n 
of V consisting of eigenvectors of S. For j e {1, let A y be the 
eigenvalue corresponding to ej . Then 

\^j\ = UjejW = \\SejW = Iky || = 1. 

Thus each eigenvalue of S has absolute value 1 , completing the proof. ■ 


EXERCISES 7.C 


1 Prove or give a counterexample: If T e C(V) is self-adjoint and there 
exists an orthonormal basis e\, ... ,e n of V such that ( Tej ,ej ) >0 for 
each j , then T is a positive operator. 

2 Suppose T is a positive operator on V. Suppose v,w € V are such that 

Tv — w and Tw — v. 

Prove that v — w. 

3 Suppose T is a positive operator on V and U is a subspace of V invariant 
under T. Prove that T\u e C(U) is a positive operator on U. 

4 Suppose T € C(V. W). Prove that T* T is a positive operator on V and 
TT* is a positive operator on W. 
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5 Prove that the sum of two positive operators on V is positive. 

6 Suppose T e C(V) is positive. Prove that T k is positive for every 
positive integer k. 

7 Suppose T is a positive operator on V. Prove that T is invertible if and 
only if 

(■ Tv,v ) > 0 

for every v e V with v ^ 0. 

8 Suppose T 6 C(V). For u, v e V, define ( u,v)t by 

( u,v)t = (Tu,v). 

Prove that (-, -)j is an inner product on V if and only if T is an invertible 
positive operator (with respect to the original inner product (■, ■)). 

9 Prove or disprove: the identity operator on F 2 has infinitely many self- 
adjoint square roots. 

10 Suppose S e C(V). Prove that the following are equivalent: 

(a) S is an isometry; 

(b) {S*u, S*v) = (w, v) for all u, v € V; 

(c) S*ei , . . . , S*e m is an orthonormal list for every orthonormal list 
of vectors e \ , . . . , e m in V; 

(d) S*ei, . . . , S*e n is an orthonormal basis for some orthonormal 
basis e \, . . . , e n of V. 

11 Suppose T\,Ti are normal operators on £(F 3 ) and both operators have 
2, 5, 7 as eigenvalues. Prove that there exists an isometry S € £(F 3 ) 
such that 7i = S*T 2 S. 

12 Give an example of two self-adjoint operators 7j, T 2 e £(F 4 ) such that 
the eigenvalues of both operators are 2,5,7 but there does not exist an 
isometry S € £(F 4 ) such that 7j = .S'* 72.5. Be sure to explain why 
there is no isometry with the required property. 

13 Prove or give a counterexample: if S € C(V) and there exists an ortho- 
normal basis e 1 ,...,e n ofV such that || Sej || = 1 for each ej , then S 
is an isometry. 

14 Let T be the second derivative operator in Exercise 21 in Section 7. A. 
Show that —Fisa positive operator. 
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Polar Decomposition and Singular 
Value Decomposition 

Polar Decomposition 

Recall our analogy between C and C{V). Under this analogy, a complex 
number z corresponds to an operator T, and z corresponds to T*. The real 
numbers (z = z) correspond to the self-adjoint operators (T — T*), and the 
nonnegative numbers correspond to the (badly named) positive operators. 

Another distinguished subset of C is the unit circle, which consists of the 
complex numbers z such that |z| = 1. The condition |z| = 1 is equivalent 
to the condition zz = 1 . Under our analogy, this would correspond to the 
condition T*T — I, which is equivalent to T being an isometry (see 7.42). 
In other words, the unit circle in C corresponds to the isometries. 

Continuing with our analogy, note that each complex number z except 0 
can be written in the form 

z = (j7j) |z| = (pj) 751 ' 

where the first factor, namely, z/|z|, is an element of the unit circle. Our 
analogy leads us to guess that each operator T e C{V) can be written as an 
isometry times JT*T . That guess is indeed correct, as we now prove after 
defining the obvious notation, which is justified by 7.36. 

7.44 Notation Vf 

If T is a positive operator, then \J~T denotes the unique positive square 
root of T. 

Now we can state and prove the Polar Decomposition, which gives a 
beautiful description of an arbitrary operator on V. Note that T*T is a 
positive operator for every T e C(V), and thus gT*T is well defined. 

7.45 Polar Decomposition 

Suppose T e C{V). Then there exists an isometry S e £(U) such that 

T = SVY*Y. 


7.D 
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Proof If v € V, then 

||7Y|| 2 = (Tv, Tv) = (T*Tv,v) 

= ( Vt*tVt*t v ,v ) 

= ( VT*Tv , VT*Tv) 

= || VT* Tv II 2 . 

Thus 

7.46 \\Tv\\ = \\VY*Tv\\ 
for all v e V. 

Define a linear map Si : range \/ T* T — >• range T by 

7.47 Si(VT*T v) = Tv. 

The idea of the proof is to extend Si to an isometry S e C(V) such that 
T — S Vr*T. Now for the details. 

First we must check that Si is well defined. To do this, suppose vi , V 2 € V 
are such that *JT*T vi = y/T*T V 2 - For the definition given by 7.47 to make 
sense, we must show that Tv\ — T V 2 - Note that 

\\Tvx-Tv 2 \\ = Iinvi-v 2 )|| 

= \\Vt^T(v 1 -v 2 )\\ 

= \\Vf*f Vl - Vt*Tv 2 || 

= 0 , 


where the second equality holds by 7.46. The equation above shows that 
Tv i = T i ; 2 , so Si is indeed well defined. You should verify that Si is a 
linear map. 

We see from 7.47 that Si maps range \/T* T onto range T. Clearly 7.46 
and 7.47 imply that 


| Si w || = || m | 


for all u € range ~JT*T . 

In particular, Si is injective. Thus 
from the Fundamental Theorem of Lin- 
ear Maps (3.22), applied to Si, we have 


The rest of the proof extends Si to 
an isometry S on all of V 


n 


dim range \/T*T — dim range T. 
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This implies that dim (range *JT*T) 1 ~ — dim(range T) 1 - (see 6.50). 

Thus orthonormal bases e\, ... ,e m of (range JT*T) 1 - and fi, . . . , f m 
of (range T )-*- can be chosen; the key point here is that these two ortho- 
normal bases have the same length (denoted m). Now define a linear map 
S 2 ■ (range y/T*T) ± —>■ (range T) 1 - by 

S 2 {aiei -\ h a m e m ) — a\f\ H b a m f m . 

For all w e (range v / 7’*r)- L , we have ||S 2 w|| = ||w|| (from 6.25). 

Now let S be the operator on V that equals 5] on range \/T*T and equals 
S 2 on (range y/T*T) ± . More precisely, recall that each v € V can be written 
uniquely in the form 

7.48 v = u + w, 

where u € range y/T*T and w € (range y/T*T) L (see 6.47). For v € V 
with decomposition as above, define Sv by 

Sv — S\u + S 2 w. 

For each v € V we have 

S(VT*Tv ) = S 1 (VT*Tv ) = Tv, 

so T = SVT*T , as desired. All that remains is to show that S is an isometry. 
Flowever, this follows easily from two uses of the Pythagorean Theorem: if 
v € V has decomposition as in 7.48, then 

||Sv|| 2 = ||Sim + 5 2 w || 2 = ||5i»|| 2 + || S 2 vi- 1| 2 = || m || 2 + || w|| 2 = ||v|| 2 ; 

the second equality holds because S 1 u e range T and ,S' 2 tv e (range 7')^. ■ 


The Polar Decomposition (7.45) states that each operator on V is the 
product of an isometry and a positive operator. Thus we can write each 
operator on V as the product of two operators, each of which comes from 
a class that we can completely describe and that we understand reasonably 
well. The isometries are described by 7.43 and 9.36; the positive operators 
are described by the Spectral Theorem (7.24 and 7.29). 

Specifically, consider the case F = C, and suppose T — S JT*T is a 
Polar Decomposition of an operator T e C(V), where S is an isometry. Then 
there is an orthonormal basis of V with respect to which 5 has a diagonal 
matrix, and there is an orthonormal basis of V with respect to which J~T*T 
has a diagonal matrix. Warning: there may not exist an orthonormal basis 
that simultaneously puts the matrices of both S and V T*T into these nice 
diagonal forms. In other words, S may require one orthonormal basis and 
may require a different orthonormal basis. 
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Singular Value Decomposition 

The eigenvalues of an operator tell us something about the behavior of the 
operator. Another collection of numbers, called the singular values, is also 
useful. Recall that eigenspaces and the notation E are defined in 5.36. 

7.49 Definition singular values 

Suppose T € C( V). The singular values of T are the eigenvalues 
of Vf*T, with each eigenvalue A repeated dim ft (A, JT*T) times. 

The singular values of T are all nonnegative, because they are the eigen- 
values of the positive operator °JT*T . 

7.50 Example Define T e £(F 4 ) by 


r(zi,z 2 ,Z 3 ,z 4 ) = (0, 3 zi,2z 2 ,-3z 4 ). 


Find the singular values of T. 

Solution A calculation shows T*T(z\, z 2 , z 3 , z 4 ) = (9zi, 4z 2 , 0, 9z 4 ), as 
you should verify. Thus 


V T*r(zi, z 2 , z 3 , z 4 ) = (3 zi,2z 2 ,0, 3z 4 ), 


and we see that the eigenvalues of J~T*T are 3,2,0 and 

dim E(3, VT*T) = 2, dim E(2, Vf*T) = 1 , dim£(0, Vf*T) = 1 . 

ffence the singular values of T are 3, 3, 2, 0. 

Note that —3 and 0 are the only eigenvalues of T. Thus in this case, the 
collection of eigenvalues did not pick up the number 2 that appears in the 
definition (and hence the behavior) of T, but the collection of singular values 
does include 2. 


Each T e C{V) has dim V singular values, as can be seen by applying 
the Spectral Theorem and 5.41 [see especially part (e)] to the positive (hence 
self-adjoint) operator \/T*T . For example, the operator T defined in Exam- 
ple 7.50 on the four-dimensional vector space F 4 has four singular values 
(they are 3, 3, 2, 0), as we saw above. 

The next result shows that every operator on V has a clean description in 
terms of its singular values and two orthonormal bases of V. 
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7.51 Singular Value Decomposition 

Suppose T e C(V) has singular values s\,...,s n . Then there exist 
orthonormal bases e\, ... ,e n and f \ of V such that 


Tv = s\(v, e\)f\ 4 b s n {v,e„)f n 


for every v e V. 


Proof By the Spectral Theorem applied to J~T*T , there is an orthonormal 

basis e\,...,e n ofV such that JT*T ej — Sjej for j = 1 n. 

We have 

v = (v, e\)e\ 4 b (v.e„)e n 

for every v € V (see 6.30). Apply JT*T to both sides of this equation, 
getting 

V T*T V = 5i(v, e\)e\ 4 b s n {v,e n )e n 

for every v € V. By the Polar Decomposition (see 7.45), there is an isometry 
S € C{V) such that T = S^/T*T. Apply S to both sides of the equation 
above, getting 

Tv = si{v,e 1 )Se 1 4 b s n (v,e n )Se n 

for every v e V. For each j, let fj — Sej. Because S is an isometry, 
/i is an orthonormal basis of V (see 7.42). The equation above now 

becomes 

Tv = si(v, e\)f\ 4 b s n (v,e n )f n 

for every v € V, completing the proof. ■ 


When we worked with linear maps from one vector space to a second 
vector space, we considered the matrix of a linear map with respect to a basis 
of the first vector space and a basis of the second vector space. When dealing 
with operators, which are linear maps from a vector space to itself, we almost 
always use only one basis, making it play both roles. 

The Singular Value Decomposition allows us a rare opportunity to make 
good use of two different bases for the matrix of an operator. To do this, 
suppose T € C(V). Let s\,...,s n denote the singular values of T, and let 
e\, ... ,e n and f \ be orthonormal bases of V such that the Singular 
Value Decomposition 7.51 holds. Because Tej — Sj fj for each j , we have 


M(T,{e l ,...,e n ),(f l /„)) 


si 0 ^ 

Sn ) 


0 
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In other words, every operator on V has a diagonal matrix with respect 
to some orthonormal bases of V, provided that we are permitted to use two 
different bases rather than a single basis as customary when working with 
operators. 

Singular values and the Singular Value Decomposition have many applica- 
tions (some are given in the exercises), including applications in computational 
linear algebra. To compute numeric approximations to the singular values of 
an operator T, first compute T* T and then compute approximations to the 
eigenvalues of T*T (good techniques exist for approximating eigenvalues 
of positive operators). The nonnegative square roots of these (approximate) 
eigenvalues of T*T will be the (approximate) singular values of T. In other 
words, the singular values of T can be approximated without computing the 
square root of T*T. The next result helps justify working with T* T instead 

of Vt*t. 

7.52 Singular values without taking square root of an operator 

Suppose T € C(V). Then the singular values of T are the nonnegative 
square roots of the eigenvalues of T* T, with each eigenvalue A repeated 
dim E( A, T*T) times. 

Proof The Spectral Theorem implies that there are an orthonormal basis 
e\, ... ,e n and nonnegative numbers Aj, . . . , X„ such that T*Tej — A jej 
for j = 1, It is easy to see that y/T*T ej = y/~h]ej for j = 1 
which implies the desired result. ■ 


EXERCISES 7.D 


1 Fix u, x e V with u ^ 0. Define T e C(V) by 

Tv — (v, u)x 


for every v € V. Prove that 

Vt*T v = J- (v, u)u 

Id'll 

for every y e V. 

2 Give an example of T e £(C 2 ) such that 0 is the only eigenvalue of T 
and the singular values of T are 5, 0. 
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3 Suppose T e C{V). Prove that there exists an isometry S G C{V) such 
that 

T = VTT*S. 

4 Suppose T g C{V) and s is a singular value of T. Prove that there exists 
a vector v G V such that ||v|| = 1 and || 7"v|| = s. 

5 Suppose T g C{ C 2 ) is defined by T(x, y ) = (—4 y, x). Find the singu- 
lar values of T. 

6 Find the singular values of the differentiation operator D g 'P(R 2 ) 
defined by Dp = p', where the inner product on 'P(R 2 ) is as in Example 
6.33. 

7 Define T € £(F 3 ) by 


T(zi,z 2 , z 3 ) = (z 3 ,2z 1 ,3z 2 ). 

Find (explicitly) an isometry S € £(F 3 ) such that T = S \/T*T . 

8 Suppose T € C(V), S e C(V) is an isometry, and R € C(V) is a 
positive operator such that T = SR. Prove that R = *JT*T . 

[The exercise above shows that if we write T as the product of an isometry 
and a positive operator {as in the Polar Decomposition 7.45), then the 
positive operator equals JT*T .] 

9 Suppose T G C( V). Prove that T is invertible if and only if there exists 
a unique isometry S € C{V ) such that T - S^T*T. 

10 Suppose T e C(V) is self-adjoint. Prove that the singular values of T 
equal the absolute values of the eigenvalues of T, repeated appropriately. 

11 Suppose T g C(V). Prove that T and T* have the same singular values. 

12 Prove or give a counterexample: if T G C( V), then the singular values 
of T 2 equal the squares of the singular values of T. 

13 Suppose T G C(V). Prove that T is invertible if and only if 0 is not a 
singular value of T. 

14 Suppose T G C{V). Prove that dimrangeT equals the number of 
nonzero singular values of T. 

15 Suppose S G C(V). Prove that S is an isometry if and only if all the 
singular values of 5 equal 1 . 
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16 Suppose T \ , T 2 g C(V). Prove that T\ and T 2 have the same singular 
values if and only if there exist isometries Si, S 2 G £■( V) such that 
T\ = SiT 2 S 2 . 

17 Suppose T G C{V) has singular value decomposition given by 


Tv = si (v, e\) f\ H h s n {v,e n )f n 


for every v G V, where si, ... ,s n are the singular values of T and 
e\, ... ,e n and f\ are orthonormal bases of V. 

(a) Prove that if y e V, then 

T*v = si{v.fi)ei H h s n {v,fn)e n . 

(b) Prove that if y e V, then 


T*Tv = s\ 2 (y, e\)e\ H h s n 2 (v,e„)e n . 


(c) Prove that if v G V, then 

s/T*Tv = si{v,e{)e\ H b s n {v,e n )e„. 


(d) Suppose T is invertible. Prove that if v G V, then 


. (v,fi)ei ( v,f n )e n 

T v — ( 1 

Sl S n 


for every y G V. 


18 Suppose T G C{V). Let .v denote the smallest singular value of T, and 
let .s' denote the largest singular value of T. 

(a) Prove that .? || v || < 1 1 7" r ’ 1 1 < 5||v|| for every v G V. 

(b) Suppose A is an eigenvalue of T. Prove that s < |A| < s. 


19 Suppose T G C(V). Show that T is uniformly continuous with respect 
to the metric d on V defined by d(u,v) = \\u — v||. 

20 Suppose S.T G C{V). Let s denote the largest singular value of S, 
let t denote the largest singular value of T, and let r denote the largest 
singular value of S + T. Prove that r < s + t. 
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Hypatia, the 5 ,h century Egyptian 
mathematician and philosopher, as 
envisioned around 1 900 by Alfred 
Seifert. 


Operators on Complex 
Vector Spaces 


In this chapter we delve deeper into the structure of operators, with most of 
the attention on complex vector spaces. An inner product does not help with 
this material, so we return to the general setting of a finite-dimensional vector 
space. To avoid some trivialities, we will assume that V V {0}- Thus our 
assumptions for this chapter are as follows: 

8.1 Notation F, V 

• F denotes R or C. 

• V denotes a finite-dimensional nonzero vector space over F. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ generalized eigenvectors and generalized eigenspaces 

■ characteristic polynomial and the Cayley-Hamilton Theorem 

■ decomposition of an operator 

■ minimal polynomial 

■ Jordan Form 
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CHAPTER 8 Operators on Complex Vector Spaces 


Generalized Eigenvectors and Nilpotent 
Operators 

Null Spaces of Powers of an Operator 

We begin this chapter with a study of null spaces of powers of an operator. 

8.2 Sequence of increasing null spaces 
Suppose T e C(V). Then 

{0} = null T° C null T 1 C ■ ■ ■ C null T k C null T k+1 C ■ ■ ■ . 


8.A 


Proof Suppose k is a nonnegative integer and v € null T k . Then T k v = 0, 
and hence T k+1 v = T{T k v) = T( 0) = 0. Thus v e null7’ fc+1 . Hence 
null T k C null T k+1 , as desired. ■ 

The next result says that if two consecutive terms in this sequence of 
subspaces are equal, then all later terms in the sequence are equal. 

8.3 Equality in the sequence of null spaces 

Suppose T € C(V). Suppose m is a nonnegative integer such that 
null T m = null T m+1 . Then 

null T m = null T m+1 = null T m+2 = null T m+ 3 = • • • . 

Proof Let A: be a positive integer. We want to prove that 
null T m+k = null T m+k+1 . 

We already know from 8.2 that null T m+k C null T m+k+l . 

To prove the inclusion in the other direction, suppose v e null T m+k+] . 
Then 

T m+1 (T k v) - T m+k+l v = 0. 

Hence 

T k v € nulir'” +1 = null T m . 

Thus T m+k v — T m (T k v) — 0, which means that v e null T m+k . This 
implies that null T m+k+1 C null T m+k , completing the proof. ■ 
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The proposition above raises the question of whether there exists a non- 
negative integer m such that null T m — null T m+l . The proposition below 
shows that this equality holds at least when m equals the dimension of the 
vector space on which T operates. 

8.4 Null spaces stop growing 
Suppose T e C{V). Let n = dim V. Then 

null T n = null T n+1 = null T n+2 = ■■■ . 

Proof We need only prove that null T n — null T' 1+1 (by 8.3). Suppose this 
is not true. Then, by 8.2 and 8.3, we have 

{0} = null T° c null T 1 c . . . c null T n c null T n+l , 

where the symbol c means “contained in but not equal to”. At each of the 
strict inclusions in the chain above, the dimension increases by at least 1 . 
Thus dim null T n+1 > n + 1, a contradiction because a subspace of V cannot 
have a larger dimension than n. m 

Unfortunately, it is not true that V = null T ©range T for each T e C( V). 
However, the following result is a useful substitute. 

8.5 V is the direct sum of null T dim v and range T dim v 
Suppose T € £(V). Let n — dim V. Then 

V = null T n © range T n . 

Proof First we show that 

8.6 (null T n ) n (range T n ) = {0}. 

Suppose v € (null T n )f 1 (range T n ). Then T n v — 0, and there exists u e V 
such that v = T n u. Applying T n to both sides of the last equation shows that 
T n v — T 2n u. Hence T 2n u = 0, which implies that T n u = 0 (by 8.4). Thus 
v — T n u — 0, completing the proof of 8.6. 

Now 8.6 implies that null T n + range T n is a direct sum (by 1.45). Also, 

dim (null T n 0 range T n ) = dim null T" + dim range T n = dim V. 

where the first equality above comes from 3.78 and the second equality comes 
from the Fundamental Theorem of Linear Maps (3.22). The equation above 
implies that null T n 0 range T n — V, as desired. ■ 


244 


CHAPTER 8 Operators on Complex Vector Spaces 


8.7 Example Suppose T € £(F 3 ) is defined by 
r(zi,z 2 ,z 3 ) = (4 z 2 ,0,5z 3 ). 

For this operator, null T + range T is not a direct sum of subspaces, because 
null T — {(zi,0, 0) : zi e F} and range T = {(zi,0, z 3 ) : zi,z 3 e F}. 
Thus null T (T range T ^ {0} and hence null T + range T is not a direct sum. 
Also note that null T + range 7 1 ^ F 3 . 

However, we have T 3 (zi, z 2 , z 3 ) = (0, 0, 125z 3 ). Thus we see that 
nulir 3 = {(zi,z 2 ,0) : zi,z 2 e F} and range T 3 = {(0, 0, z 3 ) : z 3 e F}. 
Hence F 3 = null 7” 3 © range T 3 . 


Generalized Eigenvectors 

Unfortunately, some operators do not have enough eigenvectors to lead to 
a good description. Thus in this subsection we introduce the concept of 
generalized eigenvectors, which will play a major role in our description of 
the structure of an operator. 

To understand why we need more than eigenvectors, let’s examine the 
question of describing an operator by decomposing its domain into invariant 
subspaces. Fix T e £(F). We seek to describe T by finding a “nice” direct 
sum decomposition 

V = Ui ©••• © U m , 

where each U j is a subspace of V invariant under T. The simplest possible 
nonzero invariant subspaces are 1 -dimensional. A decomposition as above 
where each Uj is a 1 -dimensional subspace of V invariant under T is possible 
if and only if V has a basis consisting of eigenvectors of T (see 5.41). This 
happens if and only if V has an eigenspace decomposition 

8.8 V = E(Xi,T)®---®E[X m ,T), 

where Ai , . . . , X m are the distinct eigenvalues of T (see 5.41). 

The Spectral Theorem in the previous chapter shows that if V is an inner 
product space, then a decomposition of the form 8.8 holds for every normal 
operator if F = C and for every self-adjoint operator if F = R because 
operators of those types have enough eigenvectors to form a basis of V (see 
7.24 and 7.29). 
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Sadly, a decomposition of the form 8.8 may not hold for more general oper- 
ators, even on a complex vector space. An example was given by the operator 
in 5.43, which does not have enough eigenvectors for 8.8 to hold. General- 
ized eigenvectors and generalized eigenspaces, which we now introduce, will 
remedy this situation. 

8.9 Definition generalized eigenvector 

Suppose T e C(V ) and A is an eigenvalue of T. A vector v e V is called 
a generalized eigenvector of T corresponding to A if v ^ 0 and 

(t — xiy v = 0 

for some positive integer j . 

Although j is allowed to be an arbi- 
trary integer in the equation 

(' T-XI) j v = 0 

in the definition of a generalized eigen- 
vector, we will soon prove that every 
generalized eigenvector satisfies this 
equation with j = dim V. 

8.10 Definition generalized eigenspace, G(X,T) 

Suppose T € C(V) and A e F. The generalized eigenspace of T corre- 
sponding to A, denoted G(A, T), is defined to be the set of all generalized 
eigenvectors of T corresponding to A, along with the 0 vector. 

Because every eigenvector of T is a generalized eigenvector of T (take 
j = 1 in the definition of generalized eigenvector), each eigenspace is 
contained in the corresponding generalized eigenspace. In other words, if 
T € C(V) and AeF, then 


Note that we do not define the con- 
cept of a generalized eigenvalue, 
because this would not lead to any- 
thing new. Reason: if (T — XI ) J is 
not injective for some positive inte- 
ger j , then T — XI is not injective, 
and hence A is an eigenvalue of T. 


E(X,T) C G(A, T). 

The next result implies that if T e C( V) and A e F, then G(A, T) is a 
subspace of V (because the null space of each linear map on V is a subspace 
of V). 
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8.1 1 Description of generalized eigenspaces 

Suppose T S C(V) and A e F. Then G(A, T) = null(T - A/) dimF 


Proof Suppose v € null(T — A/) dimF . The definitions imply v 6 G(A, T). 
Thus G(A, T) d null(T - A/) dimF 

Conversely, suppose v e G(A, T). Thus there is a positive integer j such 
that 

v e null(T — A/) 7 . 

From 8.2 and 8.4 (with T — XI replacing T), we get v e null(7' — A/) dimF . 
Thus G(A, T) C null(T — A/) dimF , completing the proof. ■ 


8.12 Example Define T e £(C 3 ) by 


T(z 1 ,z 2 ,z 3 ) = (4z 2 , 0, 5z 3 ). 

(a) Find all eigenvalues of T, the corresponding eigenspaces, and the 
corresponding generalized eigenspaces. 

(b) Show that C 3 is the direct sum of generalized eigenspaces correspond- 
ing to the distinct eigenvalues of T. 

Solution 

(a) A routine use of the definition of eigenvalue shows that the eigenvalues 
of T are 0 and 5. The corresponding eigenspaces are easily seen to be 
£(0, T) — {(zi, 0, 0) : z\ e C} and E(5 , T) — {(0, 0, z 3) : z 3 e C}. 

Note that this operator T does not have enough eigenvectors to span its 
domain C 3 . 

We have T 2 ’{z\,z 2 ,z 3 ) — (0,0, 125Z3) for all Z1.Z2.Z3 e C. Thus 
8.11 implies that G(0, T) — {(zi,Z2,0) : zi,Z2 € C}. 

We have (T — 5/) 3 (zi , z 2 , z 3 ) = (— 125zi + 300z2, — 125z2, 0). Thus 
8.11 implies that G(5, T) — {(0, 0, Z3) : Z3 e C}. 

(b) The results in part (a) show that C 3 = G(0, T) © G( 5, T ). 
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One of our major goals in this chapter is to show that the result in part (b) 
of the example above holds in general for operators on finite-dimensional 
complex vector spaces; we will do this in 8.21. 

We saw earlier (5.10) that eigenvectors corresponding to distinct eigenval- 
ues are linearly independent. Now we prove a similar result for generalized 
eigenvectors. 

8.13 Linearly independent generalized eigenvectors 

Let T e C(V). Suppose Aj X m are distinct eigenvalues of T and 

vi, . . . , v m are corresponding generalized eigenvectors. Then vi, . . . , v m 
is linearly independent. 

Proof Suppose a\, . . . ,a m are complex numbers such that 

8.14 0 = a\v\ -1 1- a m v m . 

Let k be the largest nonnegative integer such that (T — X\I) k v\ ^ 0. Let 

w = (T-Xil) k vi. 


Thus 

(T - Ai I)w — (T — X 1 I) k+l w = 0, 

and hence Tw = Aivv. Thus (T — X I)w = (Ai — A )w for every A eF and 
hence 

8.15 (T — XI) n w — (A i — X) n w 

for every AeF, where n — dim V. 

Apply the operator 

(T — X\I) k (T — X 2 I) n ■ ■ ■ (T — X m I) n 

to both sides of 8.14, getting 

0 = a 1 (T-X 1 I) k (T-X 2 I) n ---(T-X m I) n v l 
= a\(T — X 2 I) n ■■■(T — X m I) n w 
= o t (A i — X 2 ) n • ■ ■ (Ai — A m ) n w, 

where we have used 8.11 to get the first equation above and 8.15 to get the 
last equation above. 

The equation above implies that a \ = 0. In a similar fashion, aj = 0 for 
each j , which implies that v\, , v m is linearly independent. ■ 
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Nilpotent Operators 
8.16 Definition nilpotent 

An operator is called nilpotent if some power of it equals 0. 


8.17 Example nilpotent operators 

(a) The operator N e £(F 4 ) defined by 

N(z l ,z 2 ,z 3 ,z 4 ) = (z 3 ,z 4 , 0,0) 
is nilpotent because N 2 — 0. 

(b) The operator of differentiation on V m (R) is nilpotent because the 
(m + l) st derivative of every polynomial of degree at most m equals 0. 
Note that on this space of dimension m + 1, we need to raise the 
nilpotent operator to the power m + 1 to get the 0 operator. 


The Latin word nil means noth- 
ing or zero; the Latin word potent 
means power. Thus nilpotent liter- 
ally means zero power. 


The next result shows that we never 
need to use a power higher than the di- 
mension of the space. 


8.18 Nilpotent operator raised to dimension of domain is 0 
Suppose N e C{V) is nilpotent. Then fV dim ^ = 0. 

Proof Because N is nilpotent, G(0, N) — V. Thus 8.11 implies that 
null A diml/ = V, as desired. ■ 

Given an operator T on V, we want to find a basis of V such that the 
matrix of T with respect to this basis is as simple as possible, meaning that 
the matrix contains many 0’s. 

The next result shows that if N is 
nilpotent, then we can choose a basis 
of V such that the matrix of N with 
respect to this basis has more than half 
of its entries equal to 0. Later in this 
chapter we will do even better. 


If V is a complex vector space, a 
proof of the next result follows eas- 
ily from Exercise 7, 5.27, and 5.32. 
But the proof given here uses sim- 
pler ideas than needed to prove 
5.27, and it works for both real and 
complex vector spaces. 
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8.19 Matrix of a nilpotent operator 

Suppose A is a nilpotent operator on V. Then there is a basis of V with 
respect to which the matrix of A has the form 



here all entries on and below the diagonal are 0’s. 

Proof First choose a basis of null A. Then extend this to a basis of null A 1 2 . 
Then extend to a basis of null A 3 . Continue in this fashion, eventually getting 
a basis of V (because 8.18 states that null A dim1/ = V). 

Now let’s think about the matrix of A with respect to this basis. The 
first column, and perhaps additional columns at the beginning, consists of 
all 0’s, because the corresponding basis vectors are in null A. The next set 
of columns comes from basis vectors in null N 2 . Applying A to any such 
vector, we get a vector in null A; in other words, we get a vector that is a 
linear combination of the previous basis vectors. Thus all nonzero entries in 
these columns lie above the diagonal. The next set of columns comes from 
basis vectors in null A 3 . Applying A to any such vector, we get a vector in 
null A 2 ; in other words, we get a vector that is a linear combination of the 
previous basis vectors. Thus once again, all nonzero entries in these columns 
lie above the diagonal. Continue in this fashion to complete the proof. ■ 


EXERCISES 8. A 


1 Define T € £(C 2 ) by 


T(w, z ) = (z, 0). 


Find all generalized eigenvectors of T, 

2 Define T € £(C 2 ) by 


T(w,z) = (— z,w). 


Find the generalized eigenspaces corresponding to the distinct eigenval- 
ues of T. 


250 CHAPTER 8 Operators on Complex Vector Spaces 

3 Suppose T £ C(V) is invertible. Prove that G( A, T) — G(j, T~ x ) for 
every A e F with A ^ 0. 

4 Suppose T £ C(V) and a, /3 £ F with a ^ ft. Prove that 

G(a , T) n G(f. T ) = {0}. 

5 Suppose T e G(V), m is a positive integer, and v £ V is such that 
T m ~ l v ^ 0 but T m v = 0. Prove that 

v, T v, T 2 v, . . . , T m ~ l v 

is linearly independent. 

6 Suppose T e £(C 3 ) is defined by F(zi, Z 2 , Z 3 ) = (z 2 ,Z 3 , 0 ). Prove 
that T has no square root. More precisely, prove that there does not exist 
S' e £(C 3 ) such that S 2 = T. 

7 Suppose N £ C{V) is nilpotent. Prove that 0 is the only eigenvalue 
of N. 

8 Prove or give a counterexample: The set of nilpotent operators on V is a 
subspace of C{V). 

9 Suppose S,T £ C(V) and ST is nilpotent. Prove that TS is nilpotent. 

10 Suppose that T £ C(V) is not nilpotent. Let n — dim V. Show that 
V — null T n ~ l © range T n ~ x . 

11 Prove or give a counterexample: If V is a complex vector space and 
dim V — n and T £ C(V), then T n is diagonalizable. 

12 Suppose N £ C(V ) and there exists a basis of V with respect to which 
N has an upper-triangular matrix with only 0’s on the diagonal. Prove 
that N is nilpotent. 

13 Suppose V is an inner product space and N £ C{V) is normal and 
nilpotent. Prove that N = 0. 

14 Suppose V is an inner product space and N £ C{V) is nilpotent. Prove 
that there exists an orthonormal basis of V with respect to which N has 
an upper-triangular matrix. 

[If F = C, then the result above follows from Schur’s Theorem (6.38) 
without the hypothesis that N is nilpotent. Thus the exercise above needs 
to be proved only when F = R.] 
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15 Suppose TV e L{V) is such that null N AimV 1 ^ null /V dim v . Prove 
that N is nilpotent and that 

dim null N ] = j 

for every integer j with 0 < j < dim V. 

16 Suppose T € C(V). Show that 

V = range T° D range T 1 D ■■■ D range T k D range T k+1 !>■■■. 

17 Suppose T € C(V) and m is a nonnegative integer such that 

range T m = range T m+1 . 

Prove that range T k = range T m for all k > m. 

18 Suppose T e C{ V). Let n — dim V. Prove that 

range T n = range T n+1 — range T n+2 — ■■■ . 

19 Suppose T e C(V) and m is a nonnegative integer. Prove that 

null T m = null T m+ 1 if and only if range T m = range T m+ 1 . 

20 Suppose T e £(C 5 ) is such that range 7 4 ^ range T 5 . Prove that T is 
nilpotent. 

21 Find a vector space W and T € C(W) such that null T k c null T k+1 
and range T k ^ range T k+ 1 for every positive integer k. 
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Decomposition of an Operator 

Description of Operators on Complex Vector Spaces 

We saw earlier that the domain of an operator might not decompose into 
eigenspaces, even on a finite-dimensional complex vector space. In this 
section we will see that every operator on a finite-dimensional complex vector 
space has enough generalized eigenvectors to provide a decomposition. 

We observed earlier that if T e C( V), then null T and range T are invari- 
ant under T [see 5.3, parts (c) and (d)]. Now we show that the null space and 
the range of each polynomial of T is also invariant under T. 

8.20 The null space and range of p(T ) are invariant under T 

Suppose T e C{V) and p e V(F). Then null p(T) and range p{T) are 
invariant under T. 

Proof Suppose v e null p{T). Then p(T)v = 0. Thus 

{( P (T))(Tv) = T(p(T)v) = T(0) = 0. 

Hence Tv e null p(T ). Thus null p(T) is invariant under T, as desired. 

Suppose v e range p(T). Then there exists u e V such that v = p(T)u. 
Thus 

Tv = T(p(T)u) = p(T)(Tu). 

Hence Tv e range p(T). Thus range p(T) is invariant under T, as desired. ■ 

The following major result shows that every operator on a complex vector 
space can be thought of as composed of pieces, each of which is a nilpotent 
operator plus a scalar multiple of the identity. Actually we have already done 
the hai'd work in our discussion of the generalized eigenspaces G( A , T ), so at 
this point the proof is easy. 

8.21 Description of operators on complex vector spaces 

Suppose V is a complex vector space and T e C{V). Let Ai, . . . , k m be 
the distinct eigenvalues of T. Then 

(a) V = G{X\, T) © ■ ■ ■ © G(X m , T); 

(b) each G(Xj , T) is invariant under T ; 

(c) each (T — kj f)|c(Ay, 7 ’) is nilpotent. 


8.B 
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Proof Let n = dim V. Recall that G(A / , T ) — null (7' — A jl) n for each j 
(by 8.11). From 8.20 [with p(z ) = (z — A 7 ) n ], we get (b). Obviously (c) 
follows from the definitions. 

We will prove (a) by induction on n. To get started, note that the desired 
result holds if n — 1. Thus we can assume that n > 1 and that the desired 
result holds on all vector spaces of smaller dimension. 

Because V is a complex vector space, T has an eigenvalue (see 5.21); thus 
m > 1. Applying 8.5 to T — Ai/ shows that 

8.22 V = G(Ai, T) © U, 

where U = range(T — Ai /)". Using 8.20 [with p(z) = (z — Ai)”], we see 
that U is invariant under T. Because G(Ai, T) ^ {0}, we have dim U < n. 
Thus we can apply our induction hypothesis to T \ u- 

None of the generalized eigenvectors of T \ jj correspond to the eigenvalue 
Ai, because all generalized eigenvectors of T corresponding to Ai are in 
G(Ai, T). Thus each eigenvalue of T\u is in {A 2 , . . . , A m }. 

By our induction hypothesis, U = G( A 2 , T\u) © ■ ■ • © G(A m , T\u). 
Combining this information with 8.22 will complete the proof if we can show 

that G( A*., T\u) = G( A^, T) for k = 2 m. 

Thus fix k e {2, ... , m}. The inclusion G(A^, T\ jj) C G( A^, T) is clear. 
To prove the inclusion in the other direction, suppose v e G(A/,. T). By 
8.22, we can write v = vi + u, where vi e G(Ai, T) and u e U. Our 
induction hypothesis implies that 

U = V2 + ' ' ' + V m , 

where each vj is in G(Ay, T\u), which is a subset of G(A 7 -, T). Thus 


V = VI + V 2 H h Vm, 


Because generalized eigenvectors corresponding to distinct eigenvalues are 
linearly independent (see 8.13), the equation above implies that each v j equals 
0 except possibly when j — k. In particular, vi = 0 and thus v = u e U. 
Because v e U, we can conclude that v e G(A^, T\u), completing the 
proof. ■ 

As we know, an operator on a complex vector space may not have enough 
eigenvectors to form a basis of the domain. The next result shows that on a 
complex vector space there are enough generalized eigenvectors to do this. 
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8.23 A basis of generalized eigenvectors 

Suppose V is a complex vector space and T e £( V). Then there is a basis 
of V consisting of generalized eigenvectors of T. 

Proof Choose a basis of each G(Xj , T) in 8.21. Put all these bases together 
to form a basis of V consisting of generalized eigenvectors of T. m 

Multiplicity of an Eigenvalue 

If V is a complex vector space and T e C{V), then the decomposition of V 
provided by 8.21 can be a powerful tool. The dimensions of the subspaces 
involved in this decomposition are sufficiently important to get a name. 

8.24 Definition multiplicity 

• Suppose T € C{V). The multiplicity of an eigenvalue A of T 
is defined to be the dimension of the corresponding generalized 
eigenspace G(A, T). 

• In other words, the multiplicity of an eigenvalue A of T equals 
dimnullfT — A/) dimF 

The second bullet point above is justified by 8. 1 1. 

8.25 Example Suppose T e £(C 3 ) is defined by 

T(z l ,z 2 ,z 3 ) - (6zi + 3z 2 + 4z 3 ,6z 2 + 2z 3 ,7z 3 ). 

The matrix of T (with respect to the standard basis) is 



The eigenvalues of T are 6 and 7, as follows from 5.32. You can verify that 
the generalized eigenspaces of T are as follows: 

G(6, T) = span((l, 0, 0), (0, 1,0)) and G(7, T) — span((10, 2, 1)). 

Thus the eigenvalue 6 has multiplicity 2 and the eigenvalue 7 has multiplicity 1. 

The direct sum C 3 = G (6, T)®G (7, T) is the decomposition promised by 
8.21. A basis of C 3 consisting of generalized eigenvectors of T, as promised 
by 8.23, is 


( 1 , 0 , 0 ), ( 0 , 1 , 0 ), ( 10 , 2 , 1 ). 
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In Example 8.25, the sum of the multiplicities of the eigenvalues of T 
equals 3, which is the dimension of the domain of T. The next result shows 
that this always happens on a complex vector space. 

8.26 Sum of the multiplicities equals dim V 

Suppose V is a complex vector space and T e £( V). Then the sum of the 
multiplicities of all the eigenvalues of T equals dim V. 

Proof The desired result follows from 8.21 and the obvious formula for the 
dimension of a direct sum (see 3.78 or Exercise 16 in Section 2.C). ■ 

The terms algebraic multiplicity and geometric multiplicity are used in 
some books. In case you encounter this terminology, be aware that the 
algebraic multiplicity is the same as the multiplicity defined here and the 
geometric multiplicity is the dimension of the cones ponding eigenspace. In 
other words, if T e £( V) and A is an eigenvalue of T, then 

algebraic multiplicity of A = dimnull(T — A/) dim V = dimG(A, T), 
geometric multiplicity of A = dimnull(T — XI) — dim E(X, T). 

Note that as defined above, the algebraic multiplicity also has a geometric 
meaning as the dimension of a certain null space. The definition of multiplicity 
given here is cleaner than the traditional definition that involves determinants; 
10.25 implies that these definitions are equivalent. 

Block Diagonal Matrices 

To interpret our results in matrix form, 
we make the following definition, gener- 
alizing the notion of a diagonal matrix. 

If each matrix Aj in the definition 
below is a 1 -by- 1 matrix, then we actually have a diagonal matrix. 

8.27 Definition block diagonal matrix 

A block diagonal matrix is a square matrix of the form 

A i 0 \ 

0 A m J 

where A i, . . . , A m are square matrices lying along the diagonal and all 
the other entries of the matrix equal 0. 


Often we can understand a matrix 
better by thinking of it as composed 
of smaller matrices. 
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8.28 Example The 5-by-5 matrix 


/ ( 4 ) 0 0 0 0 \ 


0 / 2 -3 \ 0 0 

A= 0 \ 0 2 j 00 

0 0 0 



v 0 0 0 \ 0 1 J , 


/ 


is a block diagonal matrix with 

(A! 0 

A = A 2 

0 A 3 



where 


^, = (4). A 2 =l 2 



Here the inner matrices in the 5-by-5 matrix above are blocked off to show 
how we can think of it as a block diagonal matrix. 

Note that in the next result we get many more zeros in the matrix of T 
than are needed to make it upper triangular. 

8.29 Block diagonal matrix with upper-triangular blocks 

Suppose V is a complex vector space and T e C(V). Let Ai, . . . , X m be 

the distinct eigenvalues of T, with multiplicities d\ cl m . Then there is 

a basis of V with respect to which T has a block diagonal matrix of the 
form 


/ A 


0 \ 


\ 0 A m J 

where each A , is a dj-by-dj upper-triangular matrix of the form 
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Proof Each ( T — Xj /)|g(A 7 - ,T) is nilpotent [see 8.21(c)]. For each j, choose 
a basis of G(Xj , T), which is a vector space with dimension dj, such that the 
matrix of ( T — X j I)\ g(Xj,T) with respect to this basis is as in 8.19. Thus the 
matrix of T | G (A y ,7> which equals (T - Xj I)\ G( kj ,T) + J|g(A,-,7> with 

respect to this basis will look like the desired form shown above for Aj. 

Putting the bases of the G(Xj , T)' s together gives a basis of V [by 8.21(a)]. 
The matrix of T with respect to this basis has the desired form. ■ 

The 5-by-5 matrix in 8.28 is of the form promised by 8.29, with each of 
the blocks itself an upper-triangular matrix that is constant along the diagonal 
of the block. If T is an operator on a 5-dimensional vector space whose matrix 
is as in 8.28, then the eigenvalues of T are 4, 2, 1 (as follows from 5.32), with 
multiplicities 1, 2, 2. 

8.30 Example Suppose T e £(C 3 ) is defined by 

T(zi,z 2 ,z 3 ) - (6zi + 3z 2 + 4z 3 , 6z 2 + 2z 3 , 7z 3 ). 

The matrix of T (with respect to the standard basis) is 



which is an upper-triangular matrix but is not of the form promised by 8.29. 

As we saw in Example 8.25, the eigenvalues of T are 6 and 7 and the 
corresponding generalized eigenspaces are 


G(6, T) = span((l, 0, 0), (0, 1,0)) and G(7, T) = span((10, 2, 1)). 


We also saw that a basis of C 3 consisting of generalized eigenvectors of T is 


(1,0,0), (0,1,0), (10,2,1). 

The matrix of T with respect to this basis is 



which is a matrix of the block diagonal form promised by 8.29. 

When we discuss the Jordan Form in Section 8.D, we will see that we can 
find a basis with respect to which an operator T has a matrix with even more 
0’s than promised by 8.29. However, 8.29 and its equivalent companion 8.21 
are already quite powerful. For example, in the next subsection we will use 
8.2 1 to show that every invertible operator on a complex vector space has a 
square root. 
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Square Roots 

Recall that a square root of an operator T e C(V) is an operator R e £( V) 
such that R 2 — T (see 7.33). Every complex number has a square root, but 
not every operator on a complex vector space has a square root. For example, 
the operator on C 3 in Exercise 6 in Section 8.A has no square root. The 
noninvertibility of that operator is no accident, as we will soon see. We begin 
by showing that the identity plus any nilpotent operator has a square root. 

8.31 Identity plus nilpotent has a square root 

Suppose N € C(V) is nilpotent. Then I + N has a square root. 

Proof Consider the Taylor series for the function v 1 + x: 

8.32 VT + x = 1 + a\x + a 2 X 2 + • • • . 

We will not find an explicit formula 
for the coefficients or worry about 
whether the infinite sum converges be- 
cause we will use this equation only as 
motivation. 

Because N is nilpotent, N m — 0 for some positive integer m. In 8.32, 
suppose we replace x with N and 1 with I. Then the infinite sum on the right 
side becomes a finite sum (because N ] = 0 for all j > m). In other words, 
we guess that there is a square root of / + N of the form 

I -\- a\N -\- CI 2 N 2 u m —\N m 

Having made this guess, we can try to choose a \ , 02 such that the 

operator above has its square equal to I + N. Now 

(/ -\-ci\N -\- a 2 N~ -\- a 3 N 3 + ••• + a m —\N m 1 ) 2 

— / — 2 ct\N T - (2.02 T 0 \ 2 )N 2 T- (2r(3 T- 2o\Q2)N^ T ■ ■ ■ 

+ ( 2 a m -\ + terms involving a\,... ,a m - 2 )N m ~ l ■ 

We want the right side of the equation above to equal I + N. Hence choose a 1 
such that 2a 1 = 1 (thus a 1 = 1/2). Next, choose 02 such that 2 a 2 +a \ 2 — 0 
(thus 02 — —1/8). Then choose a 3 such that the coefficient of N 3, on the 
right side of the equation above equals 0 (thus <33 = 1/16). Continue in 
this fashion for j = 4, ... ,m — 1, at each step solving for Oj so that the 
coefficient of N J on the right side of the equation above equals 0. Actually 
we do not care about the explicit formula for the a j ’s. We need only know 
that some choice of the aj ’s gives a square root of / + N. m 


Because a \ = 1/2, the formula 
above shows that 1 + x/2 is a 
good estimate for Vl + x when x 
is small. 
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The previous lemma is valid on real and complex vector spaces. However, 
the next result holds only on complex vector spaces. For example, the operator 
of multiplication by — 1 on the 1 -dimensional real vector space R has no square 
root. 

8.33 Over C, invertible operators have square roots 

Suppose V is a complex vector space and T e C(V) is invertible. Then 

T has a square root. 

Proof Let Ai, . . . , X m be the distinct eigenvalues of T. For each j, there ex- 
ists a nilpotent operator Nj € C{G{Xj,T )) such that T\ G q_ j T) = Xjl + Nj 
[see 8.21(c)]. Because T is invertible, none of the Ay’s equals 0, so we can 
write 



for each j . Clearly Nj/Xj is nilpotent, and so I + Nj/Xj has a square root 
(by 8.31). Multiplying a square root of the complex number Ay by a square 
root of I + Nj/Xj, we obtain a square root Rj of 7" | ,T)- 
A typical vector v € V can be written uniquely in the form 


v — u i + ■ ■ • + u m , 


where each uj is in G(Ay , T) (see 8.21). Using this decomposition, define an 
operator R e C( V) by 


Rv — R i it i T • • • T Rm m 


You should verify that this operator R is a square root of T, completing the 


proof. 


By imitating the techniques in this section, you should be able to prove 
that if V is a complex vector space and T e C{V) is invertible, then T has a 
k th root for every positive integer k. 


EXERCISES 8.B 


1 Suppose V is a complex vector space, N e C(V), and 0 is the only 
eigenvalue of N. Prove that N is nilpotent. 

2 Give an example of an operator T on a finite-dimensional real vector 
space such that 0 is the only eigenvalue of T but T is not nilpotent. 
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3 Suppose T € £{V). Suppose S e £{V) is invertible. Prove that T and 
S~ l TS have the same eigenvalues with the same multiplicities. 

4 Suppose V is an n -dimensional complex vector space and T is an oper- 
ator on V such that null T n ~ 2 ^ null T n ~ l . Prove that T has at most 
two distinct eigenvalues. 

5 Suppose V is a complex vector space and T e £(V). Prove that V has 
a basis consisting of eigenvectors of T if and only if every generalized 
eigenvector of T is an eigenvector of T. 

[For F = C, the exercise above adds an equivalence to the list in 5.47.] 

6 Define N e £ (F 5 ) by 

N(x 1, *2, *3, A'4, X 5 ) = (2X2, 3X3, —X4, 4X5, 0 ). 

Find a square root of I + TV. 

7 Suppose V is a complex vector space. Prove that every invertible operator 
on V has a cube root. 


8 Suppose T e £{V) and 3 and 8 are eigenvalues of T. Let n — dim V. 
Prove that V — (null F" -2 ) © (range T n ~ 2 ). 


9 Suppose A and B are block diagonal matrices of the form 


A = 

r 

° ^ 

, B = 

< Bi 

° 


1 0 

Am y 


\ 0 

Bm ) 


where Aj has the same size as Bj for j = 1, . . . , m. Show that A B is a 
block diagonal matrix of the form 


AB 


Ai B 




10 Suppose F = C and T e £(V). Prove that there exist D, N e £{V) 
such that T = D + TV, the operator D is diagonalizable, N is nilpotent, 
and DN = ND. 

11 Suppose T € £(V) and A e F. Prove that for every basis of V with 
respect to which T has an upper-triangular matrix, the number of times 
that A appears on the diagonal of the matrix of T equals the multiplicity 
of A as an eigenvalue of T. 
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Characteristic and Minimal Polynomials 

The Cayley-Hamilton Theorem 

The next definition associates a polynomial with each operator on V if F = C. 
For F = R, the corresponding definition will be given in the next chapter. 

8.34 Definition characteristic polynomial 

Suppose V is a complex vector space and T e C{V). Let X\ X m 

denote the distinct eigenvalues of T, with multiplicities d\ d m . The 

polynomial 

(z-ki) dl -(z- X m ) dm 

is called the characteristic polynomial of T. 


8.C 


8.35 Example Suppose T e £(C 3 ) is defined as in Example 8.25. Be- 
cause the eigenvalues of T are 6, with multiplicity 2, and 7, with multiplicity 1, 
we see that the characteristic polynomial of T is (z — 6 ) 2 (z — 7). 


8.36 Degree and zeros of characteristic polynomial 
Suppose V is a complex vector space and T e C(V). Then 

(a) the characteristic polynomial of T has degree dim V ; 

(b) the zeros of the characteristic polynomial of T are the eigenvalues 
of T. 

Proof Clearly part (a) follows from 8.26 and part (b) follows from the defini- 
tion of the characteristic polynomial. ■ 

Most texts define the characteristic polynomial using determinants (the 
two definitions are equivalent by 10.25). The approach taken here, which 
is considerably simpler, leads to the following easy proof of the Cayley- 
Hamilton Theorem. In the next chapter, we will see that this result also holds 
on real vector spaces (see 9.24). 

8.37 Cayley-Hamilton Theorem 

Suppose V is a complex vector space and T e C(V). Let q denote the 
characteristic polynomial of T. Then q(T) = 0. 
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Proof Let Xi, ... , X m be the distinct 
eigenvalues of the operator T, and let 
d\. ... ,d m be the dimensions of the 
corresponding generalized eigenspaces 
G(X 1 ,T),...,G(X m ,T). For each 
j € {1 ,...,m}, we know that 

(T — X j ,T) is nilpotent. Thus 
we have (T - X jl) d ' | G(Ay . j) = 0 (by 
8.18). 

Every vector in V is a sum of vectors in G(Ai, T), . . . , G( X m . T ) (by 8.21). 
Thus to prove that q(T) = 0, we need only show that q(T)\G(kj,T) — 0 f° r 
each j . 

Thus fix j e { 1 , . . . ,m}. We have 

q(T) = (T-X 1 I) d '---(T-X m I) d "'. 

The operators on the right side of the equation above all commute, so we can 
move the factor (T — Xj I) dj to be the last term in the expression on the right. 
Because (T — Xjl) dj \g(Xj,t) = we conclude that q(T)\G(Xj ,T) = as 
desired. ■ 

The Minimal Polynomial 

In this subsection we introduce another important polynomial associated with 
each operator. We begin with the following definition. 

8.38 Definition monic polynomial 

A monic polynomial is a polynomial whose highest-degree coefficient 
equals 1. 


English mathematician Arthur 
Cayley ( 1821-1895 ) published 
three math papers before complet- 
ing his undergraduate degree in 
1842. Irish mathematician William 
Rowan Hamilton ( 1805-1865 ) was 
made a professor in 1827 when 
he was 22 years old and still an 
undergraduate! 


8.39 Example The polynomial 2 + 9 z 2 + z 7 is a monic polynomial of 
degree 7. 


8.40 Minimal polynomial 

Suppose T e C{V). Then there is a unique monic polynomial p of 
smallest degree such that p ( T ) = 0. 
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Proof Let n — dim V. Then the list 

I T T 2 T nl 

is not linearly independent in £( V), because the vector space £( V) has 
dimension n 2 (see 3.61) and we have a list of length n 2 + 1. Let m be the 
smallest positive integer such that the list 

8.41 I,T,T 2 ,...,T m 

is linearly dependent. The Linear Dependence Lemma (2.21) implies that one 
of the operators in the list above is a linear combination of the previous ones. 
Because m was chosen to be the smallest positive integer such that the list 
above is linearly dependent, we conclude that T m is a linear combination of 
I. T, T 2 , . . . , T m ~ l . Thus there exist scalars ao, a\, a 2 , . . . , a m ~\ e F such 
that 

8.42 a 0 I + a x T + a 2 T 2 + ■ ■ ■ + a m - 1 T m ~ l + T m = 0. 

Define a monic polynomial p e 'P(F) by 

p(z) — a 0 + a\z + a 2 z 1 H h a m -\z m ~ l + z m . 

Then 8.42 implies that p(T) — 0. 

To prove the uniqueness part of the result, note that the choice of m implies 
that no monic polynomial q e P(F) with degree smaller than m can satisfy 
q(T) — 0. Suppose q e P(F) is a monic polynomial with degree m and 
q(T) — 0. Then (p — q)(T) = 0 and deg(p — q) < m. The choice of m now 
implies that q = p, completing the proof. ■ 

The last result justifies the following definition. 

8.43 Definition minimal polynomial 

Suppose T e £(V). Then the minimal polynomial of T is the unique 
monic polynomial p of smallest degree such that p(T) = 0. 

The proof of the last result shows that the degree of the minimal polynomial 
of each operator on V is at most (dim V) 2 . The Cayley-Hamilton Theorem 
(8.37) tells us that if V is a complex vector space, then the minimal polynomial 
of each operator on V has degree at most dim V. This remarkable improvement 
also holds on real vector spaces, as we will see in the next chapter. 
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Suppose you are given the matrix (with respect to some basis) of an 
operator T e C(V). You could program a computer to find the minimal 
polynomial of T as follows: Consider the system of linear equations 

8.44 aoM(I) + a\M(T) + • • • + a m -\M(T) m ~ 1 = -M(T) m 

for successive values of m — 1,2,... 
until this system of equations has a solu- 
tion flo- at, < 22 , ci m -\. The scalars 

ao, fit, 02 , . . . , a m - 1 , 1 will then be the 
coefficients of the minimal polynomial of T. All this can be computed using a 
familiar and fast (for a computer) process such as Gaussian elimination. 

8.45 Example Let T be the operator on C 5 whose matrix (with respect 
to the standard basis) is 

/ 0 0 0 0 -3 \ 

1 0 0 0 6 

0 10 0 0 
0 0 10 0 

V 0 0 0 1 0 / 

Find the minimal polynomial of T. 

Solution Because of the large number of 0’s in this matrix, Gaussian elim- 
ination is not needed here. Simply compute powers of and then 

you will notice that there is clearly no solution to 8.44 until m — 5. Do 
the computations and you will see that the minimal polynomial of T equals 
z 5 - 6z + 3. 


Think of this as a system of 
(dim V ) 2 linear equations in m 
variables ao,a\, . .. 


The next result completely characterizes the polynomials that when applied 
to an operator give the 0 operator. 

8.46 q(T) — 0 implies q is a multiple of the minimal polynomial 

Suppose T € C{V) and q e V(F). Then q(T) = 0 if and only if q is a 
polynomial multiple of the minimal polynomial of T. 

Proof Let p denote the minimal polynomial of T. 

First we prove the easy direction. Suppose q is a polynomial multiple of p. 
Thus there exists a polynomial s e V(F) such that q = ps. We have 

q(T) = p(T)s{T) = 0s(T) = 0, 


as desired. 
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To prove the other direction, now suppose q(T) — 0. By the Division 
Algorithm for Polynomials (4.8), there exist polynomials s, r e 'P(F) such 
that 

8.47 q = ps + r 

and deg r < deg p. We have 

0 = q(T) = p{T)s{T) + r{T) = r(T). 

The equation above implies that r = 0 (otherwise, dividing r by its highest- 
degree coefficient would produce a monic polynomial that when applied to 
T gives 0; this polynomial would have a smaller degree than the minimal 
polynomial, which would be a contradiction). Thus 8.47 becomes the equation 
q — ps. Hence q is a polynomial multiple of p. as desired. ■ 

The next result is stated only for complex vector spaces, because we have 
not yet defined the characteristic polynomial when F = R. However, the 
result also holds for real vector spaces, as we will see in the next chapter. 

8.48 Characteristic polynomial is a multiple of minimal polynomial 

Suppose F = C and T e C(V). Then the characteristic polynomial of T 
is a polynomial multiple of the minimal polynomial of T. 

Proof The desired result follows immediately from the Cayley-Hamilton 
Theorem (8.37) and 8.46. ■ 

We know (at least when F = C) that the zeros of the characteristic 
polynomial of T are the eigenvalues of T (see 8.36). Now we show that the 
minimal polynomial has the same zeros (although the multiplicities of these 
zeros may differ). 

8.49 Eigenvalues are the zeros of the minimal polynomial 

Let T e C(V). Then the zeros of the minimal polynomial of T are 
precisely the eigenvalues of T. 

Proof Let 


p(z) = a 0 + a\z + ci 2 z 2 H b a m -\ z m 1 + z m 


be the minimal polynomial of T. 
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First suppose A e F is a zero of p. Then p can be written in the form 
p(z) = (z — X)q(z), 

where q is a monic polynomial with coefficients in F (see 4.11). Because 
p(T) - 0, we have 

0 = (T - XI)(q(T)v) 

for all v € V. Because the degree of q is less than the degree of the minimal 
polynomial p, there exists at least one vector v € V such that q(T)v ^ 0. 
The equation above thus implies that A is an eigenvalue of T, as desired. 

To prove the other direction, now suppose A e F is an eigenvalue of T. 
Thus there exists v e V with v^O such that Tv — Av. Repeated applications 
of T to both sides of this equation show that T 7 v = A 7 v for every nonnegative 
integer j . Thus 

0 = p(T)v = (a 0 I +a x T + a 2 T 2 + • • ■ + a m ^T m ~ l + T m )v 
= (flo + a iA + a 2 ^ 2 + ■ ■ • + u m —iX m 1 + A m )v 
= p(A)v. 

Because v ^ 0, the equation above implies that p{ A) = 0, as desired. ■ 

The next three examples show how our results can be useful in finding 
minimal polynomials and in understanding why eigenvalues of some operators 
cannot be exactly computed. 

8.50 Example Find the minimal polynomial of the operator T e £( C 3 ) 
in Example 8.30. 

Solution In Example 8.30 we noted that the eigenvalues of T are 6 and 7. 
Thus by 8.49, the minimal polynomial of T is a polynomial multiple of 
(z - 6)(z - 7). 

In Example 8.35, we saw that the characteristic polynomial of T is 
(z — 6 ) 2 (z — 7). Thus by 8.48 and the paragraph above, the minimal polyno- 
mial of T is either (z — 6 )(z — 7) or (z — 6 ) 2 (z — 7). A simple computation 
shows that 

(T — 6I)(T — II) + 0. 

Thus the minimal polynomial of T is (z — 6) 2 (z — 7). 
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8.51 Example Find the minimal polynomial of the operator T e £(C 3 ) 
defined by T(z\, Z 2 , z$) = (6 zi,6z2,7zj). 

Solution It is easy to see that for this operator T, the eigenvalues of T are 6 
and 7, and the characteristic polynomial of T is (z — 6 ) 2 (z — 7). 

Thus as in the previous example, the minimal polynomial of T is ei- 
ther (z — 6)(z — 7) or (z — 6) 2 (z — 7). A simple computation shows that 
(T — 6 1)(T — 71) = 0. Thus the minimal polynomial of T is (z — 6)(z — 7). 


8.52 Example What are the eigenvalues of the operator in Example 8.45? 

Solution From 8.49 and the solution to Example 8.45, we see that the 
eigenvalues of T equal the solutions to the equation 

z 5 - 6z + 3 = 0. 

Unfortunately, no solution to this equation can be computed using rational 
numbers, roots of rational numbers, and the usual rules of arithmetic (a proof 
of this would take us considerably beyond linear algebra). Thus we cannot find 
an exact expression for any eigenvalue of T in any familiar form, although 
numeric techniques can give good approximations for the eigenvalues of 
T. The numeric techniques, which we will not discuss here, show that the 
eigenvalues for this particular operator are approximately 

-1.67, 0.51, 1.40, -0.12+ 1.59/, -0.12- 1.59/. 

The nonreal eigenvalues occur as a pair, with each the complex conjugate of 
the other, as expected for a polynomial with real coefficients (see 4.15). 


EXERCISES 8.C 


1 Suppose T e £(C 4 ) is such that the eigenvalues of T are 3, 5, 8. Prove 
that (T - 3 1) 2 (T - 5/) 2 (T - SI) 2 = 0. 

2 Suppose V is a complex vector space. Suppose T e C{V) is such that 5 
and 6 are eigenvalues of T and that T has no other eigenvalues. Prove 
that ( T — 5I) n ~ l (T — 6/)” _1 = 0, where n — dim V. 

3 Give an example of an operator on C 4 whose characteristic polynomial 

equals (z — 7 ) 2 (z — 8) 2 . 
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4 Give an example of an operator on C 4 whose characteristic polyno- 
mial equals (z — l)(z — 5) 3 and whose minimal polynomial equals 
(z - 1 )(z - 5) 2 . 

5 Give an example of an operator on C 4 whose characteristic and minimal 
polynomials both equal z(z — l) 2 (z — 3). 

6 Give an example of an operator on C 4 whose characteristic polyno- 
mial equals z(z — l) 2 (z — 3) and whose minimal polynomial equals 
z(z - l)(z - 3). 

7 Suppose V is a complex vector space. Suppose T e CiV) is such that 
P 2 = P . Prove that the characteristic polynomial of P is z m (z — \ ) n , 
where m — dim null P and n — dim range P . 

8 Suppose T e C(V). Prove that T is invertible if and only if the constant 
term in the minimal polynomial of T is nonzero. 

9 Suppose T e C(V) has minimal polynomial 4+5z— 6z 2 — 7z 3 +2z 4 +z 5 . 
Find the minimal polynomial of T~ ] . 

10 Suppose V is a complex vector space and T e C(V) is invertible. 
Let p denote the characteristic polynomial of T and let q denote the 
characteristic polynomial of T~ ] . Prove that 

q(z) = -P z ^ P Q 

for all nonzero zeC. 

11 Suppose T € C( V) is invertible. Prove that there exists a polynomial 
p 6 V(F) such that T~ l = p(T). 

12 Suppose V is a complex vector space and T e C(V). Prove that V 
has a basis consisting of eigenvectors of T if and only if the minimal 
polynomial of T has no repeated zeros. 

[For complex vector spaces, the exercise above adds another equivalence 
to the list given by 5.41.] 

13 Suppose V is an inner product space and T e C(V) is normal. Prove 
that the minimal polynomial of T has no repeated zeros. 

14 Suppose V is a complex inner product space and S e C( V) is an 
isometry. Prove that the constant term in the characteristic polynomial 
of S has absolute value 1 . 


SECTION 8. C Characteristic and Minimal Polynomials 269 


15 Suppose T € C(V) and v € V. 

(a) Prove that there exists a unique monic polynomial p of smallest 
degree such that p{T)v = 0. 

(b) Prove that p divides the minimal polynomial of T. 

16 Suppose V is an inner product space and T e C{ V). Suppose 

_ i _ _ i _ _ 2 | | _/w 1 i wi 

is the minimal polynomial of T. Prove that 

a o + afz + afz 2 -\ b ci m -\z m ~ l + z m 

is the minimal polynomial of T*. 

17 Suppose F = C and T e £{V). Suppose the minimal polynomial of T 
has degree dim V. Prove that the characteristic polynomial of T equals 
the minimal polynomial of T. 

18 Suppose ao, , a n - 1 e C. Find the minimal and characteristic polyno- 
mials of the operator on C" whose matrix (with respect to the standard 
basis) is 

^ 0 — a 0 ^ 

10 — a i 

1 — a 2 

0 —a n - 2 

\ 1 —fln—l ) 

[The exercise above shows that every monic polynomial is the character- 
istic polynomial of some operator. \ 

19 Suppose V is a complex vector space and T e C{V). Suppose that 
with respect to some basis of V the matrix of T is upper triangular, with 
Ai , . . . , X n on the diagonal of this matrix. Prove that the characteristic 
polynomial of T is (z — Ai) • • ■ (z — X n ). 

20 Suppose V is a complex vector space and I 'l ..... V m are nonzero sub- 
spaces of V such that V = V\ © • • • © V m . Suppose T e C{V) and 
each V j is invariant under T. For each j , let p j denote the characteristic 
polynomial of T\vj ■ Prove that the characteristic polynomial of T equals 
Pi ••• Pm- 
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Jordan Form 


We know that if V is a complex vector space, then for every T e C(V) there 
is a basis of V with respect to which T has a nice upper-triangular matrix (see 
8 . 29 ). In this section we will see that we can do even better — there is a basis 
of V with respect to which the matrix of T contains 0 ’s everywhere except 
possibly on the diagonal and the line directly above the diagonal. 

We begin by looking at two examples of nilpotent operators. 

8.53 Example Let N e £(F 4 ) be the nilpotent operator defined by 
N(zi,z 2 ,z 3 ,z 4 ) = (0,z 1 ,z 2 ,z 3 ). 

If v = ( 1 , 0 , 0 , 0 ), then N 3 v, N 2 v, Nv, v is a basis of F 4 . The matrix of TV 
with respect to this basis is 

/ 0 1 0 0 \ 

0 0 10 

0 0 0 1 

\ 0 0 0 0 / 

The next example of a nilpotent operator has more complicated behavior 
than the example above. 


8.D 


8.54 Example Let N e £(F 6 ) be the nilpotent operator defined by 
N (z \ , z 2 , z 3 , z 4 , z 5 , z 6 ) = (0, z\, z 2 , 0, z 4 , 0). 


Unlike the nice behavior of the nilpotent operator of the previous exam- 
ple, for this nilpotent operator there does not exist a vector v e F 6 such 
that N 5 v, N 4 v, N 3 v, N 2 v, Nv,v is a basis of F 6 . However, if we take 


vi = (1,0, 0,0, 0,0), i'2 = (0, 0, 0, 1, 0, 0), and V3 = (0, 0, 0, 0, 0, 1), then 
N 2 \’i , Nv \ , vi , Nv 2 , V2, V3 is a basis of F 6 . The matrix of N with respect to 


this basis is 


/ 


V 


0 1 0 \ 
0 0 1 
0 0 0 / 
0 0 0 
0 0 0 
0 0 0 


0 0 0 \ 

0 0 0 

0 0 0 



0 0 ( 0 ) / 


Here the inner matrices are blocked off to show that we can think of the 6-by-6 
matrix above as a block diagonal matrix consisting of a 3 -by -3 block with l’s 
on the line above the diagonal and 0’s elsewhere, a 2-by-2 block with 1 above 
the diagonal and 0’s elsewhere, and a 1-by-l block containing 0. 
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Our next result shows that every nilpotent operator N € C(V) behaves 
similarly to the previous example. Specifically, there is a finite collection of 
vectors v\, ... ,v n e V such that there is a basis of V consisting of the vectors 
of the form N k \’j, as j varies from 1 to n and k varies (in reverse order) from 
0 to the largest nonnegative integer nij such that N mj vj ^ 0. For the matrix 
interpretation of the next result, see the first part of the proof of 8.60. 

8.55 Basis corresponding to a nilpotent operator 

Suppose N € C,( V) is nilpotent. Then there exist vectors vi , . . . , v n e V 

and nonnegative integers mi, ... , m n such that 

(a) N mi v i, . . . , JVvi, vi, . . . , N m,1 v n Nv n ,v n is abasis of V; 

(b) N mi+1 V! = ••• = N m » +1 v n = 0. 

Proof We will prove this result by induction on dim V. To get started, note 
that the desired result obviously holds if dim V = 1 (in that case, the only 
nilpotent operator is the 0 operator, so take v\ to be any nonzero vector and 
m i = 0). Now assume that dim V > I and that the desired result holds on all 
vector spaces of smaller dimension. 

Because N is nilpotent, N is not injective. Thus N is not surjective (by 
3.69) and hence range fV is a subspace of V that has a smaller dimension 
than V. Thus we can apply our induction hypothesis to the restriction operator 
N | range N e £ (range IV). | We can ignore the trivial case range N = {0}, 
because in that case N is the 0 operator and we can choose vj , . . . , v n to be 
any basis of V and m\ = ■ ■ ■ = m n — 0 to get the desired result.] 

By our induction hypothesis applied to /V| rangeJ v, there exist vectors 
vi, ... ,v n € range N and nonnegative integers mi, ... , m n such that 

8.56 N mi v i, . . . , JVvi, vi, . . . , N mn v n , . . . , Nv n ,v n 

is a basis of range N and 

N mi+1 v i = ••• = N m ' , + 1 v n = 0. 

Because each vj is in range N, for each j there exists uj e V such 
that Vj = Nuj. Thus N k+ 1 ii j = N k Vj for each j and each nonnegative 
integer k. We now claim that 


8.57 


N mi+1 ui , . . . , Nui,ui , .... N mn + l u n , . . . , Nu n , u n 
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is a linearly independent list of vectors in V. To verify this claim, suppose 
that some linear combination of 8.57 equals 0. Applying N to that linear 
combination, we get a linear combination of 8.56 equal to 0. However, the 
list 8.56 is linearly independent, and hence all the coefficients in our original 
linear combination of 8.57 equal 0 except possibly the coefficients of the 
vectors 

N mi + 1 u u ...,N m " +1 u n , 

which equal the vectors 

N mi vi , .... N m,1 v n - 

Again using the linear independence of the list 8.56, we conclude that those 
coefficients also equal 0, completing our proof that the list 8.57 is linearly 
independent. 

Now extend 8.57 to a basis 

8.58 N mi + 1 ui, . . . , Nu\,u \, . . . , N m,1 + l u n Nu n ,u n ,w\, ... ,w p 

of V (which is possible by 2.33). Each Nwj is in range N and hence is in the 
span of 8.56. Each vector in the list 8.56 equals N applied to some vector in 
the list 8.57. Thus there exists xj in the span of 8.57 such that Nwj = Nxj. 
Now let 

u n+j — w j ~ *j ■ 

Then Nu n +j = 0. Furthermore, 

N mi + 1 U!, . . . , Nu\,ui, . . . , N mn+1 u„ — ,Nu n ,u n ,u n+ i,...,u n + p 

spans V because its span contains each xj and each n n +j and hence each Wj 
(and because 8.58 spans V). 

Thus the spanning list above is a basis of V because it has the same length 
as the basis 8.58 (where we have used 2.42). This basis has the required form, 
completing the proof. ■ 


In the next definition, the diagonal of 
each Aj is filled with some eigenvalue 
Xj of T. the line directly above the di- 
agonal of Aj is filled with l’s, and all 
other entries in Aj are 0 (to understand why each Xj is an eigenvalue of T, 
see 5.32). The Xj’s need not be distinct. Also, Aj may be a 1-by-l matrix 
(Xj ) containing just an eigenvalue of T. 


French mathematician Camille Jor- 
dan (1838-1922) first published a 
proof of 8.60 in 1870. 
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8.59 Definition Jordan basis 


Suppose T e C(V). A basis of V is called a Jordan basis for T if with 
respect to this basis T has a block diagonal matrix 


( A1 

0 > 

{ 0 

Ap j 


where each Aj is an upper-triangular matrix of the form 


/ Xj 1 0 \ 



1 

^■j j 


8.60 Jordan Form 

Suppose V is a complex vector space. If T € C( V), then there is a basis 
of V that is a Jordan basis for T. 

Proof First consider a nilpotent operator N € C(V) and the vectors 
vi , . . . , v n € V given by 8.55. For each j , note that N sends the first vector 
in the list N mj vj , ... , Nvj , vj to 0 and that N sends each vector in this list 
other than the first vector to the previous vector. In other words, 8.55 gives a 
basis of V with respect to which N has a block diagonal matrix, where each 
matrix on the diagonal has the form 

/ 0 1 0 \ 

•• 1 

V° 0 / 

Thus the desired result holds for nilpotent operators. 

Now suppose T e C{V). Let A | , . . . , X m be the distinct eigenvalues of T. 
We have the generalized eigenspace decomposition 

where each (T —X j I )\q^ / .T) i s nilpotent (see 8.21). Thus some basis of each 
G(Xj , T) is a Jordan basis for (T — X j I )|o’(A„ 7 , 7 ’) (see previous paragraph). 
Put these bases together to get a basis of V that is a Jordan basis for T. m 
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EXERCISES 8.D 


1 Find the characteristic polynomial and the minimal polynomial of the 
operator N in Example 8.53. 

2 Find the characteristic polynomial and the minimal polynomial of the 
operator N in Example 8.54. 

3 Suppose N € C(V) is nilpotent. Prove that the minimal polynomial of 
N is z m+l , where m is the length of the longest consecutive string of 
l’s that appears on the line directly above the diagonal in the matrix of 
N with respect to any Jordan basis for N. 

4 Suppose T e C{V) and vi , . . . , v n is a basis of V that is a Jordan basis 
for T. Describe the matrix of T with respect to the basis v n , ... , v\ 
obtained by reversing the order of the v’s. 

5 Suppose T s C{V) and v\, ... ,v n is a basis of V that is a Jordan basis 
for T. Describe the matrix of T 2 with respect to this basis. 

6 Suppose N € C{V) is nilpotent and vi , . . . , v n and m i , . . . , m n are as 

in 8.55. Prove that N mi v i N mn v n is a basis of null N. 

[The exercise above implies that n, which equals dim null /V, depends 
only on N and not on the specific Jordan basis chosen for N.] 

7 Suppose p,q € V(C) are monic polynomials with the same zeros and q 
is a polynomial multiple of p. Prove that there exists T e £(C deg? ) such 
that the characteristic polynomial of T is q and the minimal polynomial 
of T is p. 

8 Suppose V is a complex vector space and T e C(V). Prove that there 
does not exist a direct sum decomposition of V into two proper subspaces 
invariant under T if and only if the minimal polynomial of T is of the 
form (z — A) dim v for some A e C. 
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geometry (from The 
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painted by Raphael 
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Operators on Real Vector Spaces 


In the last chapter we learned about the structure of an operator on a finite- 
dimensional complex vector space. In this chapter, we will use our results 
about operators on complex vector spaces to learn about operators on real 
vector spaces. 

Our assumptions for this chapter are as follows: 

9.1 Notation F, V 

• F denotes R or C. 

• V denotes a finite-dimensional nonzero vector space over F. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ complexification of a real vector space 

■ complexification of an operator on a real vector space 

■ operators on finite-dimensional real vector spaces have an 
eigenvalue or a 2-dimensional invariant subspace 

■ characteristic polynomial and the Cayley-Hamilton Theorem 

■ description of normal operators on a real inner product space 

■ description of isometries on a real inner product space 
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Complexification 

Complexification of a Vector Space 

As we will soon see, a real vector space V can be embedded, in a natural way, 
in a complex vector space called the complexification of V. Each operator 
on V can be extended to an operator on the complexification of V. Our 
results about operators on complex vector spaces can then be translated to 
information about operators on real vector spaces. 

We begin by defining the complexification of a real vector space. 

9.2 Definition complexification of V, V c 
Suppose V is a real vector space. 

• The complexification of V. denoted Vc , equals V x V. An element 
of V c is an ordered pair (m, v), where u, v € V, but we will write 
this as u + i v. 

• Addition on Vc is defined by 

(mi + ivi) + (n 2 + iv 2 ) = (mi + m 2 ) + i(v\ + v 2 ) 
for Mi, vi, m 2 , v 2 € V. 

• Complex scalar multiplication on Vc is defined by 

( 1 a + bi)(u + iv) = (an — bv) + i (av + bu) 
for a, b eR and u , v e V. 

Motivation for the definition above of complex scalar multiplication comes 
from usual algebraic properties and the identity i 2 = — 1 . If you remember 
the motivation, then you do not need to memorize the definition above. 

We think of V as a subset of Vc by identifying u € V with u + i 0. 
The construction of Vc from V can then be thought of as generalizing the 
construction of C” from R" . 

9.3 Vc is a complex vector space. 

Suppose V is a real vector space. Then with the definitions of addition 
and scalar multiplication as above, Vc is a complex vector space. 

The proof of the result above is left as an exercise for the reader. Note that 
the additive identity of Vc is 0 + i 0, which we write as just 0. 


9.A 
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Probably everything that you think should work concerning complexifica- 
tion does work, usually with a straightforward verification, as illustrated by 
the next result. 

9.4 Basis of V is basis of V c 
Suppose V is a real vector space. 

(a) If i’i , . . . , v n is a basis of V (as a real vector space), then v\, ... ,v n 
is a basis of Vq (as a complex vector space). 

(b) The dimension of Vq (as a complex vector space) equals the dimen- 
sion of V (as a real vector space). 

Proof To prove (a), suppose vi v n is a basis of the real vector space V. 

Then span(vi , . . . , v n ) in the complex vector space Vq contains all the vectors 

vi , . . . , v n , i vi i v n . Thus v\, ... ,v n spans the complex vector space Vq ■ 

To show that v\. ... ,v n is linearly independent in the complex vector 
space Vq , suppose X\, ... ,X n e C and 


At vi + • • ■ + A n v n — 0. 

Then the equation above and our definitions imply that 

(ReAi)vi H 1- (Re X n )v n = 0 and (ImAi)vi H h (ImA„)v n = 0. 

Because iq, . . . , v n is linearly independent in V, the equations above imply 
ReAi = ■■■ = ReA„ = 0 and ImAi = ■■■ = Im„ = 0. Thus we have 
X\ — ■■■ — X n — 0. Hence vi,...,v„ is linearly independent in Vq, 
completing the proof of (a). 

Clearly (b) follows immediately from (a). ■ 

Complexification of an Operator 

Now we can define the complexification of an operator. 

9.5 Definition complexification of T, Tq 

Suppose V is a real vector space and T e C(V). The complexification of 
T, denoted Tq, is the operator Tq e C{V c) defined by 

Tq (u + i v) = Tu + iTv 


for u, v € V. 
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You should verify that if V is a real vector space and T e C(V), then Tq 
is indeed in C(Vq). The key point here is that our definition of complex scalar 
multiplication can be used to show that 7c(A(w + tv)) = A 7c (u + zv) for 
all w, v e V and all complex numbers A. 

The next example gives a good way to think about the complexification of 
a typical operator. 


9.6 Example Suppose A is an n-by-n matrix of real numbers. Define 
T € £(R”) by Tx — Ax, where elements of R" are thought of as n-by-1 
column vectors. Identifying the complexification of R' ? with C”, we then 
have Tqz = Az for each z e C n , where again elements of C” are thought of 
as n-by-1 column vectors. 

In other words, if T is the operator of matrix multiplication by A on R”, 
then the complexification Tq is also matrix multiplication by A but now acting 
on the larger domain C” . 

The next result makes sense because 9.4 tells us that a basis of a real vector 
space is also a basis of its complexification. The proof of the next result 
follows immediately from the definitions. 

9.7 Matrix of T c equals matrix of T 

Suppose V is a real vector space with basis vi, . . . ,v n and T e C(V). 

Then M.{T) = M.(Tq), where both matrices are with respect to the basis 

vi , • • • , v„. 

The result above and Example 9.6 provide complete insight into complexi- 
fication, because once a basis is chosen, every operator essentially looks like 
Example 9.6. Complexification of an operator could have been defined using 
matrices, but the approach taken here is more natural because it does not 
depend on the choice of a basis. 

We know that every operator on a nonzero finite-dimensional complex 
vector space has an eigenvalue (see 5.21) and thus has a 1-dimensional in- 
variant subspace. We have seen an example [5.8(a)] of an operator on a 
nonzero finite-dimensional real vector space with no eigenvalues and thus no 
1 -dimensional invariant subspaces. However, we now show that an invariant 
subspace of dimension 1 or 2 always exists. Notice how complexification 
leads to a simple proof of this result. 
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9.8 Every operator has an invariant subspace of dimension 1 or 2 

Every operator on a nonzero finite-dimensional vector space has an 
invariant subspace of dimension 1 or 2. 

Proof Every operator on a nonzero finite-dimensional complex vector space 
has an eigenvalue (5.21) and thus has a 1-dimensional invariant subspace. 

Hence assume V is a real vector space and T e C(V). The complexifica- 
tion 7c has an eigenvalue a + hi (by 5.21), where a.h e R . Thus there exist 
u, v € V, not both 0, such that 7c (u + iv) — {a + bi)(u + iv). Using the 
definition of 7c , the last equation can be rewritten as 

Tu + iTv — ( au — bv) + ( av + bu)i. 

Thus 

Tu = au — bv and T v — av + bu. 

Let U equal the span in V of the list u , v. Then U is a subspace of V 
with dimension 1 or 2. The equations above show that U is invariant under T, 
completing the proof. ■ 

The Minimal Polynomial of the Complexification 

Suppose V is a real vector space and T e C(V). Repeated application of the 
definition of 7c shows that 

9.9 (T c ) n (u + iv) = T n u + iT n v 

for every positive integer n and all u,v e V. 

Notice that the next result implies that the minimal polynomial of 7c has 
real coefficients. 

9.10 Minimal polynomial of T c equals minimal polynomial of T 

Suppose V is a real vector space and T e C(V). Then the minimal 
polynomial of Tq equals the minimal polynomial of T. 

Proof Let p e 'P ( R ) denote the minimal polynomial of T. From 9.9 it is 
easy to see that p(Tq) = ( p(T)) c , and thus p(Tc) — 0. 

Suppose q € V( C) is a monic polynomial such that q(Tc) — 0. Then 
(q(Tc))(u) = 0 for every u € V. Letting r denote the polynomial whose j lh 
coefficient is the real part of the j th coefficient of q, we see that r is a monic 
polynomial and r(T ) = 0. Thus deg q — deg r > deg p. 

The conclusions of the two previous paragraphs imply that p is the minimal 
polynomial of Tq , as desired. ■ 
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Eigenvalues of the Complexification 

Now we turn to questions about the eigenvalues of the complexification of an 
operator. Again, everything that we expect to work indeed works easily. 

We begin with a result showing that the real eigenvalues of Tq are precisely 
the eigenvalues of T. We give two different proofs of this result. The first 
proof is more elementary, but the second proof is shorter and gives some 
useful insight. 

9 . 1 1 Real eigenvalues of T c 

Suppose V is a real vector space, T e C(V), and A e R. Then A is an 
eigenvalue of Tq if and only if A is an eigenvalue of T. 

Proof 1 First suppose A is an eigenvalue of T. Then there exists v e V 
with v / 0 such that Tv = Av. Thus 7c v = Av, which shows that A is an 
eigenvalue of Tq , completing one direction of the proof. 

To prove the other direction, suppose now that A is an eigenvalue of Tq ■ 
Then there exist u, v e V with u + i v / 0 such that 

7c (m + iv) — A(m + tv). 

The equation above implies that T u — An and Tv — Av. Because w / 0 or 
v ^ 0, this implies that A is an eigenvalue of T, completing the proof. ■ 

Proof 2 The (real) eigenvalues of T are the (real) zeros of the minimal 
polynomial of T (by 8.49). The real eigenvalues of Tq are the real zeros of the 
minimal polynomial of 7c (again by 8.49). These two minimal polynomials 
are the same (by 9.10). Thus the eigenvalues of T are precisely the real 
eigenvalues of 7c , as desired. ■ 

Our next result shows that Tq behaves symmetrically with respect to an 
eigenvalue A and its complex conjugate A. 

9.12 Tq — A I and Tq — A I 

Suppose V is a real vector space, T € C(V), A G C, j is a nonnegative 
integer, and u , v € V. Then 

(Tq — A/) 7 (u + iv) = 0 if and only if (7c — A7 ) J ' (u — iv) = 0. 
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Proof We will prove this result by induction on j . To get started, note that 
if j = 0 then (because an operator raised to the power 0 equals the identity 
operator) the result claims that u + iv = 0 if and only if u — iv — 0, which is 
clearly true. 

Thus assume by induction that j > 1 and the desired result holds for j — 1. 
Suppose (7c — A7 ) 7 (m + iv) = 0. Then 

9.13 (7c — A/) J-1 ((7c — A7)(u + iv)) =0. 

Writing A = a + bi, where a, b € R, we have 

9.14 (7c — A7)(w + iv) = (T u — au + bv) + i(Tv — av — bu ) 
and 

9.15 (7c — A7)(w — iv) = ( T u — au + bv) — i(Tv — av — bu). 

Our induction hypothesis, 9.13, and 9.14 imply that 

(7c — A7) 7-1 (( T u — au + bv) — i (T v — av — bu)) = 0. 

Now the equation above and 9.15 imply that (7c — A /) 7 (u — iv) = 0, 
completing the proof in one direction. 

The other direction is proved by replacing A with A, replacing v with — v, 
and then using the first direction. ■ 

An important consequence of the result above is the next result, which 
states that if a number is an eigenvalue of 7c, then its complex conjugate is 
also an eigenvalue of Tq . 

9.16 Nonreal eigenvalues of T c come in pairs 

Suppose V is a real vector space, T e C{V), and A e C. Then A is an 
eigenvalue of Tq if and only if A is an eigenvalue of Tq . 

Proof Take j — 1 in 9.12. ■ 

By definition, the eigenvalues of an operator on a real vector space are 
real numbers. Thus when mathematicians sometimes informally mention the 
complex eigenvalues of an operator on a real vector space, what they have in 
mind is the eigenvalues of the complexification of the operator. 

Recall that the multiplicity of an eigenvalue is defined to be the dimension 
of the generalized eigenspace corresponding to that eigenvalue (see 8.24). The 
next result states that the multiplicity of an eigenvalue of a complexification 
equals the multiplicity of its complex conjugate. 
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9.17 Multiplicity of A equals multiplicity of A 

Suppose V is a real vector space, T e C(V), and A e C is an eigenvalue 
of Tq. Then the multiplicity of A as an eigenvalue of Tq equals the 
multiplicity of A as an eigenvalue of Tq ■ 

Proof Suppose u\ + iv\,...,u m + iv m is a basis of the generalized 

eigenspace G(A, 7c), where u\ ,u m ,v i v m e V. Then using 9.12, 

routine arguments show that u\ — iv \, . . . , u m — iv m is a basis of the gen- 
eralized eigenspace G(A, 7c). Thus both A and A have multiplicity m as 
eigenvalues of 7c . ■ 


9.18 Example Suppose T e £(R 3 ) is defined by 


T(x i,x 2 ,x 3 ) = (2x\ , x 2 -x 3 ,x 2 + x 3 ). 



2 0 
0 1 
0 1 


The matrix of T with respect to the standard basis of R 3 is 


As you can verify, 2 is an eigenvalue of T with multiplicity 1 and T has no 
other eigenvalues. 

If we identify the complexification of R 3 with C 3 , then the matrix of Tq 
with respect to the standard basis of C 3 is the matrix above. As you can 
verify, the eigenvalues of 7c are 2,1 + i, and 1 — i, each with multiplicity 
1 . Thus the nonreal eigenvalues of Tq come as a pair, with each the complex 
conjugate of the other and with the same multiplicity, as expected by 9.17. 

We have seen an example [5.8(a)] of an operator on R 2 with no eigenvalues. 
The next result shows that no such example exists on R 3 . 

9.19 Operator on odd-dimensional vector space has eigenvalue 
Every operator on an odd-dimensional real vector space has an eigenvalue. 

Proof Suppose V is a real vector space with odd dimension and T e C( V). 
Because the nonreal eigenvalues of 7c come in pairs with equal multiplicity 
(by 9.17), the sum of the multiplicities of all the nonreal eigenvalues of 7c is 
an even number. 

Because the sum of the multiplicities of all the eigenvalues of Tq equals 
the (complex) dimension of Vc (by Theorem 8.26), the conclusion of the 
paragraph above implies that Tq has a real eigenvalue. Every real eigenvalue 
of 7c is also an eigenvalue of T (by 9.1 1), giving the desired result. ■ 
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Characteristic Polynomial of the Complexification 

In the previous chapter we defined the characteristic polynomial of an operator 
on a finite-dimensional complex vector space (see 8.34). The next result is 
a key step toward defining the characteristic polynomial for operators on 
finite-dimensional real vector spaces. 

9.20 Characteristic polynomial of T c 

Suppose V is a real vector space and T e C(V). Then the coefficients of 
the characteristic polynomial of Tq are all real. 

Proof Suppose A is a nonreal eigenvalue of Tq with multiplicity m. Then A is 
also an eigenvalue of 7c with multiplicity m (by 9.17). Thus the characteristic 
polynomial of Tq includes factors of (z — A)'” and (z — X) m . Multiplying 
together these two factors, we have 

(z — A) m (z — X) m = (z 2 — 2 (Re A)z + |A| 2 ) m . 

The polynomial above on the right has real coefficients. 

The characteristic polynomial of Tq is the product of terms of the form 
above and terms of the form (z — t) d , where t is a real eigenvalue of 7c with 
multiplicity d. Thus the coefficients of the characteristic polynomial of Tq 
are all real. ■ 

Now we can define the characteristic polynomial of an operator on a 
finite-dimensional real vector space to be the characteristic polynomial of its 
complexification. 

9.21 Definition Characteristic polynomial 

Suppose V is a real vector space and T e C(V). Then the characteristic 
polynomial of T is defined to be the characteristic polynomial of 7c - 


9.22 Example Suppose?" e £(R 3 ) is defined by 

T{X 1 ,X 2 ,X 3 ) = (2x 1 , X '2 - x 3 ,x 2 + x 3 ). 

As we noted in 9.18, the eigenvalues of 7c are 2, 1 + /, and 1 — i, each with 
multiplicity 1 . Thus the characteristic polynomial of the complexification 7c 
is (z — 2)(z — (1 + i))(z — (1 — i )), which equals z 3 — 4z 2 + 6z — 4. Hence 
the characteristic polynomial of T is also z 3 — 4z 2 + 6z — 4. 
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In the next result, the eigenvalues of T are all real (because T is an operator 
on a real vector space). 

9.23 Degree and zeros of characteristic polynomial 
Suppose V is areal vector space and T e £( V). Then 

(a) the coefficients of the characteristic polynomial of T are all real; 

(b) the characteristic polynomial of T has degree dim V ; 

(c) the eigenvalues of T are precisely the real zeros of the characteristic 
polynomial of T. 

Proof Part (a) holds because of 9.20. 

Part (b) follows from 8.36(a). 

Part (c) holds because the real zeros of the characteristic polynomial of T 
are the real eigenvalues of Tq [by 8.36(a)], which are the eigenvalues of T 
(by 9.11). ■ 

In the previous chapter, we proved the Cayley-Hamilton Theorem (8.37) 
for complex vector spaces. Now we can also prove it for real vector spaces. 

9.24 Cayley-Hamilton Theorem 

Suppose T e C(V). Let q denote the characteristic polynomial of T. 
Then q(T) = 0. 

Proof We have already proved this result when V is a complex vector space. 
Thus assume that V is a real vector space. 

The complex case of the Cayley-Hamilton Theorem (8.37) implies that 
<7 (7c) = 0. Thus we also have q(T) = 0, as desired. ■ 


9.25 Example Supposed e £(R 3 ) is defined by 

T(x 1,X 2 ,X3) - (2*1, *2 -X 3 ,X 2 + x 3 ). 

As we saw in 9.22, the characteristic polynomial of T is z 3 — 4z 2 + 6z — 4. 
Thus the Cayley-Hamilton Theorem implies that T 3 — AT 2 + 6T — Al — 0, 
which can also be verified by direct calculation. 

We can now prove another result that we previously knew only in the 
complex case. 
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9.26 Characteristic polynomial is a multiple of minimal polynomial 
Suppose T € C{V). Then 

(a) the degree of the minimal polynomial of T is at most dim V ; 

(b) the characteristic polynomial of T is a polynomial multiple of the 
minimal polynomial of T. 

Proof Part (a) follows immediately from the Cayley-Hamilton Theorem. 
Part (b) follows from the Cayley-Hamilton Theorem and 8.46. ■ 


EXERCISES 9. A 


1 Prove 9.3. 

2 Verify that if V is a real vector space and T e C{V), then 7c € C{Vq). 

3 Suppose V is a real vector space and vi v m € V. Prove that 

vi , . . . , v m is linearly independent in Vq if and only if v\, ... ,v m is 
linearly independent in V. 

4 Suppose V is a real vector space and vi , . . . , v m e V. Prove that 

vi, ... ,v m spans Vq if and only if vj , . . . , v m spans V. 

5 Suppose that V is a real vector space and S, T e C( V). Show that 

( S + T ) c = Sc + 7c ar *d that (A 79c = A 7c for every A e R. 

6 Suppose V is a real vector space and T e £{V). Prove that Tq is 
invertible if and only if T is invertible. 

7 Suppose V is a real vector space and N e £( V). Prove that Nq is 
nilpotent if and only if N is nilpotent. 

8 Suppose T € £(R 3 ) and 5, 7 are eigenvalues of T. Prove that 7c has no 
nonreal eigenvalues. 

9 Prove there does not exist an operator T e £(R 7 ) such that T 2 + T + 7 
is nilpotent. 

10 Give an example of an operator T e £(C 7 ) such that T 2 + T + 7 
is nilpotent. 
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11 Suppose V is a real vector space and T € C(V). Suppose there exist 
b, c € R such that T 2 + bT + cl — 0. Prove that T has an eigenvalue 
if and only if b 2 > 4c. 

12 Suppose V is a real vector space and T € C(V). Suppose there exist 
b, c € R such that b 2 < 4c and T 2 + bT + cl is nilpotent. Prove that 
T has no eigenvalues. 

13 Suppose V is a real vector space, T e C{V), and b, c € R are such that 
b 2 < 4c. Prove that null (T 2 + bT + cl) 1 has even dimension for every 
positive integer j . 

14 Suppose V is a real vector space with dim V = 8. Suppose T e C[V) 
is such that T 2 + T + / is nilpotent. Prove that (T 2 + T + /) 4 = 0. 

15 Suppose V is a real vector space and T e C(V) has no eigenvalues. 
Prove that every subspace of V invariant under T has even dimension. 

16 Suppose V is a real vector space. Prove that there exists T e C(V) such 
that T 2 — —I if and only if V has even dimension. 

17 Suppose V is a real vector space and T e C(V) satisfies T 2 = —I. 
Define complex scalar multiplication on V as follows: if a, b e R, then 

(a + bi)v — av + bTv. 

(a) Show that the complex scalar multiplication on V defined above 
and the addition on V makes V into a complex vector space. 

(b) Show that the dimension of V as a complex vector space is half 
the dimension of V as a real vector space. 

18 Suppose V is a real vector space and T e C( V). Prove that the following 
are equivalent: 

(a) All the eigenvalues of Tq are real. 

(b) There exists a basis of V with respect to which T has an upper- 
triangular matrix. 

(c) There exists a basis of V consisting of generalized eigenvectors 
of T. 

19 Suppose V is a real vector space with dim V — n and T e C(V) is 
such that null T n ~ 2 ^ null T n ~ l . Prove that T has at most two distinct 
eigenvalues and that 7c has no nonreal eigenvalues. 
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Operators on Real Inner Product Spaces 

We now switch our focus to the context of inner product spaces. We will give 
a complete description of normal operators on real inner product spaces; a 
key step in the proof of this result (9.34) requires the result from the previous 
section that an operator on a finite-dimensional real vector space has an 
invariant subspace of dimension 1 or 2 (9.8). 

After describing the normal operators on real inner product spaces, we will 
use that result to give a complete description of isometries on such spaces. 

Normal Operators on Real Inner Product Spaces 

The Complex Spectral Theorem (7.24) gives a complete description of normal 
operators on complex inner product spaces. In this subsection we will give a 
complete description of normal operators on real inner product spaces. 

We begin with a description of the operators on 2-dimensional real inner 
product spaces that are normal but not self-adjoint. 

9.27 Normal but not self-adjoint operators 

Suppose V is a 2-dimensional real inner product space and T e C( V). 
Then the following are equivalent: 

(a) T is normal but not self-adjoint. 

(b) The matrix of T with respect to 
the form 

( a 

b 

with 1) / 0. 

(c) The matrix of T with respect to 
the form 

( a 

b 

with b > 0. 

Proof First suppose (a) holds, so that T is normal but not self-adjoint. Let 
e\, e 2 be an orthonormal basis of V. Suppose 

M(T,(eue 2 )) = [l j). 


every orthonormal basis of V has 



some orthonormal basis of V has 



9.B 


9.28 
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Then ||r<?i|| 2 = a 2 + b 2 and ||r*ei|| 2 = a 2 + c 2 . Because T is normal, 
|| 7>i || = || T*e 1 1| (see 7.20); thus these equations imply that b 2 = c 2 . Thus 
c = b or c = —b. But c ^ b, because otherwise T would be self-adjoint, as 
can be seen from the matrix in 9.28. Hence c = —b, so 

9.29 M{T,( ei ,e 2 )) = ( a b ) • 

The matrix of T* is the transpose of the matrix above. Use matrix multipli- 
cation to compute the matrices of T T* and T* T (do it now). Because T is 
normal, these two matrices are equal. Equating the entries in the upper-right 
corner of the two matrices you computed, you will discover that bd = ab. 
Now b ^ 0, because otherwise T would be self-adjoint, as can be seen from 
the matrix in 9.29. Thus d — a, completing the proof that (a) implies (b). 

Now suppose (b) holds. We want to prove that (c) holds. Choose an 
orthonormal basis e \ , e?_ of V. We know that the matrix of T with respect to 
this basis has the form given by (b), with b ^ 0. If b > 0, then (c) holds 
and we have proved that (b) implies (c). If b < 0, then, as you should verify, 
the matrix of T with respect to the orthonormal basis ej , — <?2 equals ( b ), 
where — b > 0; thus in this case we also see that (b) implies (c). 

Now suppose (c) holds, so that the matrix of T with respect to some 
orthonormal basis has the form given in (c) with b > 0. Clearly the matrix 
of T is not equal to its transpose (because b ^ 0). Hence T is not self-adjoint. 
Now use matrix multiplication to verify that the matrices of T T* and T* T 
are equal. We conclude that TT* = T* T. Hence T is normal. Thus (c) 
implies (a), completing the proof. ■ 

The next result tells us that a normal operator restricted to an invariant 
subspace is normal. This will allow us to use induction on dim V when we 
prove our description of normal operators (9.34). 

9.30 Normal operators and invariant subspaces 

Suppose V is an inner product space, T e C(V) is normal, and U is a 

subspace of V that is invariant under T. Then 

(a) C/- 1 - is invariant under T ; 

(b) U is invariant under T*\ 

(c) ( T\ v )* = (T*)\u; 

(d) T\u G C(U) and T\ v ± € C(U^) are normal operators. 
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Proof First we will prove (a). Let e\,...,e m be an orthonormal basis 
of U. Extend to an orthonormal basis e\, . ■ ■ ,e m , fi, . . . , fn of V (this is 
possible by 6.35). Because U is invariant under T, each Tej is a linear 

combination of e\ e m . Thus the matrix of T with respect to the basis 

e\ , e m , /i , . . . , f n is of the form 

^1 ■ • • f\ ■ ■ ■ fn 


e\ ( 


A 


B 


\ 


M(T) 


h 

fn \ 


0 


c 


/ 


here A denotes an m-by-m matrix, 0 denotes the n-by-m matrix of all 0’s, B 
denotes an m-by-n matrix, C denotes an n-by-n matrix, and for convenience 
the basis has been listed along the top and left sides of the matrix. 

For each j e {1, . . . , m}, [| Tej || 2 equals the sum of the squares of the 
absolute values of the entries in the j th column of A (see 6.25). Hence 


9.31 


E 


7 = 1 



the sum of the squares of the absolute 
values of the entries of A. 


For each j e {1, . . . ,m}, ||'T*e / -|| 2 equals the sum of the squares of the 
absolute values of the entries in the j th rows of A and B. Hence 


m 

9.32 Ell^H 2 

7 = 1 

Because T is normal, 


the sum of the squares of the absolute 
values of the entries of A and B . 

Tej || = \\T*ej\\ for each j (see 7.20); thus 


E 


7 = 1 



E 


7 = 1 



This equation, along with 9.31 and 9.32, implies that the sum of the squares 
of the absolute values of the entries of B equals 0. In other words, B is the 
matrix of all 0’s. Thus 


290 


CHAPTER 9 Operators on Real Vector Spaces 


C 1 ■ • ■ £m fl ■ ■ ■ fn 

\ 

A 0 


0 C 

This representation shows that Tfa is in the span of f\ f n for each k. 

Because j\, . . . , f n is a basis of £/-*-, this implies that Tv e U 1 - whenever 
v e (V. In other words, U 1 - is invariant under T, completing the proof of (a). 

To prove (b), note that A4 (T*), which is the conjugate transpose ol\K4( T), 
has a block of 0’s in the lower left corner (because M (T), as given above, has 
a block of 0’s in the upper right corner). In other words, each T*ej can be 
written as a linear combination of e \ , . . . , e m . Thus U is invariant under T*, 
completing the proof of (b). 

To prove (c), let S = T\u 6 C(U). Fix v e U. Then 

( Su,v ) — ( Tu,v ) 

= (: u,T*v ) 

for all u € U. Because T*v € U [by (b)], the equation above shows that 
S*v — T*v. In other words, (T\jj)* — completing the proof of (c). 

To prove (d), note that T commutes with T* (because T is normal) and 
that (7V)* = (T*)\jj [by (c)]. Thus T\u commutes with its adjoint and 
hence is normal. Interchanging the roles of U and (V, which is justified by 
(a), shows that T\jj± is also normal, completing the proof of (d). ■ 

Our next result shows that normal 
operators on real inner product spaces 
come close to having diagonal matrices. 
Specifically, we get block diagonal ma- 
trices, with each block having size at 
most 2-by-2. 

We cannot expect to do better than the next result, because on a real inner 
product space there exist normal operators that do not have a diagonal matrix 
with respect to any basis. For example, the operator T e £(R 2 ) defined by 
T(x, y ) = (— v, x ) is normal (as you should verify) but has no eigenvalues; 
thus this particular T does not have even an upper-triangular matrix with 
respect to any basis of R 2 . 


Note that if an operator T has a 
block diagonal matrix with respect 
to some basis, then the entry in 
each \-by-\ block on the diagonal 
of this matrix is an eigenvalue of T. 


9.33 


M(T) = 


ei 


Sm 

fl 


fn 
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9.34 Characterization of normal operators when F = R 

Suppose V is a real inner product space and T e £(V). Then the follow- 
ing are equivalent: 

(a) T is normal. 

(b) There is an orthonormal basis of V with respect to which T has a 
block diagonal matrix such that each block is a 1 -by- 1 matrix or a 
2-by-2 matrix of the form 



with b > 0. 

Proof First suppose (b) holds. With respect to the basis given by (b), the 
matrix of T commutes with the matrix of T* (which is the transpose of the 
matrix of T), as you should verify (use Exercise 9 in Section 8.B for the 
product of two block diagonal matrices). Thus T commutes with T*, which 
means that T is normal, completing the proof that (b) implies (a). 

Now suppose (a) holds, so T is normal. We will prove that (b) holds 
by induction on dim V. To get started, note that our desired result holds if 
dim V = 1 (trivially) or if dim V = 2 [if T is self-adjoint, use the Real 
Spectral Theorem (7.29); if T is not self-adjoint, use 9.27], 

Now assume that dim V > 2 and that the desired result holds on vector 
spaces of smaller dimension. Let U be a subspace of V of dimension 1 that 
is invariant under T if such a subspace exists (in other words, if T has an 
eigenvector, let U be the span of this eigenvector). If no such subspace exists, 
let U be a subspace of V of dimension 2 that is invariant under T (an invariant 
subspace of dimension 1 or 2 always exists by 9.8). 

If dim (7 = 1, choose a vector in U with norm 1 ; this vector will be 
an orthonormal basis of U, and of course the matrix of T\u e £(U) is a 
1-by-l matrix. If dim U — 2, then T\u e £(U) is normal (by 9.30) but not 
self-adjoint (otherwise T\jj, and hence T, would have an eigenvector by 7.27). 
Thus we can choose an orthonormal basis of U with respect to which the 
matrix of T\jj € £(U) has the required form (see 9.27). 

Now U 1 - is invariant under T and T\y± is a normal operator on U 1 - 
(by 9.30). Thus by our induction hypothesis, there is an orthonormal basis 
of U 1 - with respect to which the matrix of T\jj± has the desired form. Adjoin- 
ing this basis to the basis of U gives an orthonormal basis of V with respect 
to which the matrix of T has the desired form. Thus (b) holds. ■ 
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Isometries on Real Inner Product Spaces 

As we will see, the next example is a key building block for isometries on real 
inner product spaces. Also, note that the next example shows that an isometry 
on R 2 may have no eigenvalues. 

9.35 Example Let 9 € R. Then the operator on R 2 of counterclockwise 
rotation (centered at the origin) by an angle of 9 is an isometry, as is geomet- 
rically obvious. The matrix of this operator with respect to the standard basis 
is 

( cos 9 — sin 9 
sin 9 cos 9 

If 6 is not an integer multiple of n, then no nonzero vector of R 2 gets mapped 
to a scalar multiple of itself, and hence the operator has no eigenvalues. 

The next result shows that every isometry on a real inner product space is 
composed of pieces that are rotations on 2-dimensional subspaces, pieces that 
equal the identity operator, and pieces that equal multiplication by — 1 . 

9.36 Description of isometries when F = R 

Suppose V is a real inner product space and S e C(V). Then the following 
are equivalent: 

(a) S is an isometry. 

(b) There is an orthonormal basis of V with respect to which S has 
a block diagonal matrix such that each block on the diagonal is a 
1-by-l matrix containing 1 or — 1 or is a 2-by-2 matrix of the form 

( cos 9 — sin 9 
sin 9 cos 9 

with 9 € (0, tv ) . 

Proof First suppose (a) holds, so S is an isometry. Because S is normal, there 
is an orthonormal basis of V with respect to which S has a block diagonal 
matrix such that each block is a 1 -by- 1 matrix or a 2-by-2 matrix of the form 





with b > 0 (by 9.34). 
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If A is an entry in a 1 -by- 1 matrix along the diagonal of the matrix of S 
(with respect to the basis mentioned above), then there is a basis vector ej 
such that Sej — A ej. Because S is an isometry, this implies that |A| = I . 
Thus A = 1 or A = — 1 , because these are the only real numbers with absolute 
value 1. 

Now consider a 2-by-2 matrix of the form 9.37 along the diagonal of the 
matrix of S . There are basis vectors ej , ej + 1 such that 

Sej — aej + bej+ 1 . 

Thus 

1 = Iky || 2 = || Sej || 2 = a 2 + b 2 . 

The equation above, along with the condition b > 0, implies that there exists 
a number 9 e (0, re) such that a — cos 9 and b — sin 9. Thus the matrix 9.37 
has the required form, completing the proof in this direction. 

Conversely, now suppose (b) holds, so there is an orthonormal basis of V 
with respect to which the matrix of S has the form required by the theorem. 
Thus there is a direct sum decomposition 

V = lh ©•••© U m , 


where each U j is a subspace of V of dimension 1 or 2. Furthermore, any two 
vectors belonging to distinct U’s are orthogonal, and each S \ u ■ is an isometry 
mapping Uj into Uj . I f v e V, we can write 

V = U !+••• + llm, 

where each Uj is in Uj. Applying S to the equation above and then taking 
norms gives 

||Sv|| 2 = 


Sui + ••• + Su m \\ 

Su 1 1 | 2 + ■ ■ ■ + || Su m | 

2 i i II,, n 2 


| U j 


+ M/i 


V 


Thus S is an isometry, and hence (a) holds. 
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EXERCISES 9.B 


1 Suppose S € £(R 3 ) is an isometry. Prove that there exists a nonzero 
vector x € R 3 such that S 2 x — x. 

2 Prove that every isometry on an odd-dimensional real inner product space 
has 1 or — 1 as an eigenvalue. 

3 Suppose V is a real inner product space. Show that 

{u + z'v, x + iy) — (u.x) + (v, y) + ((v, x) - (u, y))i 
for u,v,x,y e V defines a complex inner product on Vq- 

4 Suppose V is a real inner product space and T e C(V ) is self-adjoint. 
Show that 7c is a self-adjoint operator on the inner product space Vq 
defined by the previous exercise. 

5 Use the previous exercise to give a proof of the Real Spectral Theorem 
(7.29) via complexification and the Complex Spectral Theorem (7.24). 

6 Give an example of an operator T on an inner product space such that T 
has an invariant subspace whose orthogonal complement is not invariant 
under T. 

[The exercise above shows that 9.30 can fail without the hypothesis that 
T is normal .] 

7 Suppose T € C(V) and T has a block diagonal matrix 



with respect to some basis of V. For j = I .... , in, let Tj be the operator 
on V whose matrix with respect to the same basis is a block diagonal 
matrix with blocks the same size as in the matrix above, with Aj in the 
j th block, and with all the other blocks on the diagonal equal to identity 
matrices (of the appropriate size). Prove that T = T\ ■ ■ ■ T m . 

8 Suppose D is the differentiation operator on the vector space V in 
Exercise 21 in Section 7. A. Find an orthonormal basis of V such that 
the matrix of the normal operator D has the form promised by 9.34. 
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British mathematician and pioneer 
computer scientist Ada Lovelace 
(1815-1852), as painted by Alfred 
Chalon in this 1840 portrait. 


Trace and Determinant 


Throughout this book our emphasis has been on linear maps and operators 
rather than on matrices. In this chapter we pay more attention to matrices as 
we define the trace and determinant of an operator and then connect these 
notions to the corresponding notions for matrices. The book concludes with 
an explanation of the important role played by determinants in the theory of 
volume and integration. 

Our assumptions for this chapter are as follows: 
io.i Notation F, V 

• F denotes R or C. 

• V denotes a finite-dimensional nonzero vector space over F. 


LEARNING OBJECTIVES FOR THIS CHAPTER 

■ change of basis and its effect upon the matrix of an operator 

■ trace of an operator and of a matrix 

■ determinant of an operator and of a matrix 

■ determinants and volume 
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Trace 


For our study of the trace and determinant, we will need to know how the 
matrix of an operator changes with a change of basis. Thus we begin this 
chapter by developing the necessary material about change of basis. 

Change of Basis 

With respect to every basis of V, the matrix of the identity operator I e C{V) 
is the diagonal matrix with l’s on the diagonal and 0’s elsewhere. We also use 
the symbol / for the name of this matrix, as shown in the next definition. 

10.2 Definition identity matrix, I 

Suppose n is a positive integer. The n-by-n diagonal matrix 

1 0 \ 

0 1 

is called the identity matrix and is denoted I. 

Note that we use the symbol / to denote the identity operator (on all vector 
spaces) and the identity matrix (of all possible sizes). You should always be 
able to tell from the context which particular meaning of I is intended. For 
example, consider the equation AA(I) = /; on the left side I denotes the 
identity operator, and on the right side I denotes the identity matrix. 

If A is a square matrix (with entries in F, as usual) with the same size as I, 
then AI — I A = A, as you should verify. 

10.3 Definition invertible, inverse, A~ ] 

A square matrix A is called invertible if there is a square matrix B of 
the same size such that AB = BA = /; we call B the inverse of A and 
denote it by A -1 . 


10.A 


Some mathematicians use the 
terms nonsingular , which means 
the same as invertible, and 
singular , which means the same 
as noninvertible. 


The same proof as used in 3.54 
shows that if A is an invertible square 
matrix, then there is a unique matrix B 
such that AB — BA = 1 (and thus the 
notation B = A _1 is justified). 
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In Section 3.C we defined the matrix of a linear map from one vector space 
to another with respect to two bases — one basis of the first vector space and 
another basis of the second vector space. When we study operators, which are 
linear maps from a vector space to itself, we almost always use the same basis 
for both vector spaces (after all, the two vector spaces in question are equal). 
Thus we usually refer to the matrix of an operator with respect to a basis and 
display at most one basis because we are using one basis in two capacities. 

The next result is one of the unusual cases in which we use two different 
bases even though we have operators from a vector space to itself. It is just a 
convenient restatement of 3.43 (with U and W both equal to V), but now we 
are being more careful to include the various bases explicitly in the notation. 
The result below holds because we defined matrix multiplication to make it 
true — see 3.43 and the material preceding it. 

10.4 The matrix of the product of linear maps 

Suppose u\, , u n and vi , . . . , v„ and w\, . . . ,w n are all bases of V. 
Suppose S.T € C(V). Then 

M(ST, (ui,...,u n ),(wi,...,w n )) = 

M(S , (vi, . . . ,v„), (w i, . . .,w n ))M(T. (mi, . . . ,u n ), (vi v n )). 

The next result deals with the matrix of the identity operator 7 with 
respect to two different bases. Note that the k th column of the matrix 

M.(I , (m i u n ), (vi, . . . , v n )) consists of the scalars needed to write u ^ 

as a linear combination of vi v n . 

10.5 Matrix of the identity with respect to two bases 

Suppose Mi,..., u n and vi, . . . , v„ are bases of V. Then the matrices 

M(l, (ui,...,u n ),(vi v„)) and M(l, (vi, . . . , v„), (mi, . . . , u n )) 

are invertible, and each is the inverse of the other. 

Proof In 10.4, replace Wj with Uj, and replace S and T with 7, getting 
7 = M(l. (vi, . . . , v„), (mi, . . . ,u n ))M(l,(u\, . . . , u n ), (v! — , v„)). 
Now interchange the roles of the n’s and v’s, getting 
7 = M(I, (mi Un), (vi, . . . ,v„))A4(7, (vi, . . . ,v„), (mi, . . . , u n )). 


These two equations give the desired result. 
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10.6 Example Consider the bases (4, 2), (5, 3) and (1, 0), (0, 1) of F 2 . 
Obviously 

M(l, ((4, 2), (5, 3)), ((1,0), (0,1))) = ( £ 3 )- 

because 7(4,2) = 4(1,0) + 2(0, 1) and 7(5,3) = 5(1,0) + 3(0, 1). 

The inverse of the matrix above is 


as you should verify. Thus 10.5 implies that 

M(l, (d,0), (0,1)). ((4,2), (5.3))) = 


Now we can see how the matrix of T changes when we change bases. In 
the result below, we have two different bases of V. Recall that the notation 
M.(T, {u\, . . . , u n j) is shorthand for M(T, (u\, . . . , u n ), (u\, . . . , u n )) 

10.7 Change of basis formula 

Suppose T e C(V). Let u \, . . . , u n and vi, . . . , v n be bases of V. Let 
A = M(l, (wi,...,m„),(vi,...,v„)). Then 

M{T,(ui,...,u n )) = A- l M{T,(yi,...,v n ))A. 

Proof In 10.4, replace Wj with uj and replace S with 7, getting 
10.8 M(T, (Mi, . . . , U n )) - A~ 1 M(T, (Ml, . . . , U n ), (vi, . . . , V n )), 
where we have used 10.5. 

Again use 10.4, this time replacing Wj with vj. Also replace T with 7 and 
replace S with T, getting 

M(T, (mi,. . . ,u n ), (vi, . . . ,v n )) = M(T , (vi v„))A. 



Substituting the equation above into 10.8 gives the desired result. 
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Trace: A Connection Between Operators and Matrices 

Suppose T € C( V) and A is an eigenvalue of T. Let n = dim V. Re- 
call that we defined the multiplicity of A to be the dimension of the gen- 
eralized eigenspace G(X.T ) (see 8.24) and that this multiplicity equals 
dimnull(T — A /)” (see 8.11). Recall also that if V is a complex vector 
space, then the sum of the multiplicities of all the eigenvalues of T equals n 
(see 8.26). 

In the definition below, the sum of the eigenvalues “with each eigenvalue 
repeated according to its multiplicity” means that if A i, , X m are the distinct 
eigenvalues of T (or of Tq if V is a real vector space) with multiplicities 
d \ , . . . , d m , then the sum is 


d\X\ + •■■-(- dmX 


Or if you prefer to list the eigenvalues with each repeated according to its 
multiplicity, then the eigenvalues could be denoted X\, ... ,X n (where the 
index n equals dim V) and the sum is 


Ai + • • • + X n 


10.9 Definition trace of an operator 
Suppose T e C(V). 

• If F = C, then the trace of T is the sum of the eigenvalues of T, 
with each eigenvalue repeated according to its multiplicity. 

• If F = R, then the trace of T is the sum of the eigenvalues of 7c, 
with each eigenvalue repeated according to its multiplicity. 

The trace of T is denoted by trace T. 


10.10 Example Suppose T e £(C 3 ) is the operator whose matrix is 



Then the eigenvalues of T are 1, 2 + 3 i, and 2 — 3/, each with multiplicity 1, 
as you can verify. Computing the sum of the eigenvalues, we find that 
trace T — 1 + (2 + 3/) + (2 — 3/); in other words, trace T — 5. 
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The trace has a close connection with the characteristic polynomial. Sup- 
pose Xi, ... ,X n are the eigenvalues of T (or of Tq if V is a real vector space) 
with each eigenvalue repeated according to its multiplicity. Then by definition 
(see 8.34 and 9.21), the characteristic polynomial of T equals 


(z — Xi) ••• (z — X n ). 


Expanding the polynomial above, we can write the characteristic polynomial 
of T in the form 

1 0.1 1 z n — (Ai + • • • + X n )z n 1 + • • • + (— l)”(Ai ■ ■ ■ Xn). 

The expression above immediately leads to the following result. 

10.12 Trace and characteristic polynomial 

Suppose T e C( V). Let n = dim V. Then trace T equals the negative of 
the coefficient of r” _1 in the characteristic polynomial of T. 

Most of the rest of this section is devoted to discovering how to compute 
trace T from the matrix of T (with respect to an arbitrary basis). 

Let’s start with the easiest situation. Suppose V is a complex vector space, 
T € C{V), and we choose a basis of V as in 8.29. With respect to that basis, 
T has an upper-triangular matrix with the diagonal of the matrix containing 
precisely the eigenvalues of T, each repeated according to its multiplicity. 
Thus trace T equals the sum of the diagonal entries of M.(T) with respect to 
that basis. 

The same formula works for the operator T e £(C 3 ) in Example 10.10 
whose trace equals 5. In that example, the matrix is not in upper-triangular 
form. However, the sum of the diagonal entries of the matrix in that example 
equals 5, which is the trace of the operator T. 

At this point you should suspect that trace T equals the sum of the diagonal 
entries of the matrix of T with respect to an arbitrary basis. Remarkably, this 
suspicion turns out to be true. To prove it, we start by making the following 
definition. 

10.13 Definition trace of a matrix 

The trace of a square matrix A, denoted trace A, is defined to be the sum 
of the diagonal entries of A. 
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Now we have defined the trace of an operator and the trace of a square 
matrix, using the same word “trace” in two different contexts. This would be 
bad terminology unless the two concepts turn out to be essentially the same. 
As we will see, it is indeed true that trace T — trace Ml ( T, (vi, . . . , v n ) ) , 
where vi , . . . , v n is an arbitrary basis of V. We will need the following result 
for the proof. 

10.14 Trace of AB equals trace of BA 

If A and B are square matrices of the same size, then 

trace (A 5) = trace (B A). 

Proof Suppose 


/ M,i ■ 

A\, n ^ 

’ 5 = 

( B U1 . 

B\,n ^ 

V A n,l 

An,n y 


V B n , 1 

Bn,n / 


The j th term on the diagonal of A B equals 

n 

J 2 A hk R k,j- 
k= 1 


Thus 


n n 

trace (A 5) = EE A j,k B kj 

7 = 1 k = 1 
n n 

= B k,i A i,k 

k=lj = 1 
n 

— k lh term on the diagonal of BA 
k= 1 

= trace (2T4), 


as desired. ■ 

Now we can prove that the sum of the diagonal entries of the matrix of 
an operator is independent of the basis with respect to which the matrix is 
computed. 
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10.15 Trace of matrix of operator does not depend on basis 

Let T e C{V). Suppose u\,...,u n and vi ,v n are bases of V. Then 

trace Ad (T, (u\, . . . , u n )) = trace M(T, (vi v„)). 

Proof Let A — M(I ,(u\ u n ), (vi, . . . , v n )). Then 

trace M(T, (u \ u n )) = trace (^A ~ 1 (M(T, (vi, . . . , v n ))A) j 

= trace ^ (Ad (T, (vi, . . . , v n ))A)A~ 1 ^ 

= trace Ad (T, (vi v n )), 

where the first equality comes from 10.7 and the second equality follows 
from 10.14. The third equality completes the proof. ■ 

The result below, which is the most important result in this section, states 
that the trace of an operator equals the sum of the diagonal entries of the 
matrix of the operator. This theorem does not specify a basis because, by the 
result above, the sum of the diagonal entries of the matrix of an operator is 
the same for every choice of basis. 

10.16 Trace of an operator equals trace of its matrix 
Suppose T € C(V). Then trace T = trace M(T). 

Proof As noted above, trace A4 (T) is independent of which basis of V we 
choose (by 10.15). Thus to show that 

traceT = trace Ad (T) 

for every basis of V, we need only show that the equation above holds for 
some basis of V. 

As we have already discussed, if V is a complex vector space, then choos- 
ing the basis as in 8.29 gives the desired result. If V is a real vector space, 
then applying the complex case to the complexification Tq (which is used to 
define trace T) gives the desired result. ■ 

If we know the matrix of an operator on a complex vector space, the result 
above allows us to find the sum of all the eigenvalues without finding any of 
the eigenvalues, as shown by the next example. 
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1 0.1 7 Example Consider the operator on C 5 whose matrix is 

/ 0 0 0 0 -3 \ 

1 0 0 0 6 

0 10 0 0 
0 0 10 0 

v 0 0 0 1 0 / 

No one can find an exact formula for any of the eigenvalues of this operator. 
However, we do know that the sum of the eigenvalues equals 0, because the 
sum of the diagonal entries of the matrix above equals 0. 

We can use 10.16 to give easy proofs of some useful properties about 
traces of operators by shifting to the language of traces of matrices, where 
certain properties have already been proved or are obvious. The proof of the 
next result is an example of this technique. The eigenvalues of S + T are not, 
in general, formed from adding together eigenvalues of S and eigenvalues of 
T. Thus the next result would be difficult to prove without using 10. 16. 

10.18 Trace is additive 

Suppose S,T € C(V). Then trace (S + T) = trace S + trace T. 

Proof Choose a basis of V. Then 

traceCS + T) = trace M.(S + T) 

— trace(Ad(S) + M(T )) 

= trace Ad (S') + trace Ad (T) 

= trace S + trace T, 

where again the first and last equalities come from 10.16; the third equality is 
obvious from the definition of the trace of a matrix. ■ 

The techniques we have developed 
have the following curious consequence. 

A generalization of this result to infinite- 
dimensional vector spaces has impor- 
tant consequences in modern physics, 
particularly in quantum theory. 


The statement of the next result 
does not involve traces, although 
the short proof uses traces. When- 
ever something like this happens in 
mathematics, we can be sure that 
a good definition lurks in the back- 
ground. 
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10.19 The identity is not the difference of ST and TS 
There do not exist operators S,T e £{ V ) such that ST — TS = I. 

Proof Suppose S,T € C{V). Choose a basis of V . Then 

trace (ST — TS) = trace(ST) — trac e(TS) 

= trace M(ST) — trace M (T S) 

= trace (M(S)M(T)) — trace (Ad (T) Ad (S)) 

= 0 , 

where the first equality comes from 10.18, the second equality comes from 
10.16, the third equality comes from 3.43, and the fourth equality comes from 
10.14. Clearly the trace of / equals dim V, which is not 0. Because ST — TS 
and / have different traces, they cannot be equal. ■ 


EXERCISES 10. A 


1 Suppose T e C(V) and vi, . . . , v„ is a basis of V. Prove that the matrix 
Ad (T, (vi , . . . , v„)) is invertible if and only if T is invertible. 

2 Suppose A and B are square matrices of the same size and AB — I. 
Prove that BA = I. 

3 Suppose T e £(V ) has the same matrix with respect to every basis of V. 
Prove that T is a scalar multiple of the identity operator. 

4 Suppose Mi, . . . , u n and vi, . . . , v n are bases of V. Let T e C(V) be the 
operator such that Tv^ — for k = 1 . Prove that 

Ad (T, (vi,...,v„)) = Ad(7, (m ! , . . . ,u n ), (Vi, . . . ,v„)). 

5 Suppose B is a square matrix with complex entries. Prove that there 
exists an invertible square matrix A with complex entries such that 
A~ l BA is an upper-triangular matrix. 

6 Give an example of a real vector space V and T e £{V) such that 
trace(T 2 3 4 5 6 7 ) < 0. 

7 Suppose V is a real vector space, T e £(F), and V has a basis consisting 
of eigenvectors of T. Prove that trace (T 2 ) > 0. 


8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 
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Suppose V is an inner product space and v, w € V. Define T € £{ V) by 
Tu — (u,v)w. Find a formula for trace T. 

Suppose P e C{V) satisfies P 2 = P . Prove that 

trace P = dim range P. 

Suppose V is an inner product space and T e £( V). Prove that 


trace T* = trace T. 


Suppose V is an inner product space. Suppose T e £( V) is a positive 
operator and trace T = 0. Prove that T — 0. 

Suppose V is an inner product space and P, Q e C(V) are orthogonal 
projections. Prove that trace (P <9 ) > 0. 

Suppose T € £(C 3 ) is the operator whose matrix is 



51 -12 -21 
60 -40 -28 
57 -68 1 


Someone tells you (accurately) that —48 and 24 are eigenvalues of T. 
Without using a computer or writing anything down, find the third eigen- 
value of T. 

Suppose T e £(V) and c e F. Prove that trace (cT) — c trace T. 

Suppose S,T € £{V). Prove that trace(ST) = trac e(TS). 

Prove or give a counterexample: if S, T e C( F), then trace(ST) = 
(trace S') (trace T). 

Suppose T e £(V) is such that trace(SP) = 0 for all S € £(V). Prove 
that T - 0. 

Suppose V is an inner product space with orthonormal basis e\, ... ,e„ 
and T e £{V). Prove that 


trace(P*r) = ||rci|| 2 + ••• + ||7>„|| 2 


Conclude that the right side of the equation above is independent of 
which orthonormal basis e\, ... ,e n is chosen for V. 
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19 Suppose V is an inner product space. Prove that 

(S,T) = trac e(ST*) 
defines an inner product on C(V). 

20 Suppose V is a complex inner product space and T e C( V). Let 
X\, ... ,X n be the eigenvalues of T, repeated according to multiplicity. 
Suppose 

^ ^4 i,t • • • M ^ 

\ A n> i . . ■ A n ^ n J 

is the matrix of T with respect to some orthonormal basis of V. Prove 
that 

\X\\ 2 + ■ ■ ■ + \X n \ 2 < ^ '^2 \ Ajkl 2 . 

k= 1 j = 1 

21 Suppose V is an inner product space. Suppose T e C(V) and 

||r*v|| < || t v || 

for every v e V. Prove that T is normal. 

| The exercise above fails on infinite-dimensional inner product spaces, 
leading to what are called hyponormal operators, which have a well- 
developed theory .] 
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Determinant of an Operator 

Now we are ready to define the determinant of an operator. Notice that the 
definition below mimics the approach we took when defining the trace, with 
the product of the eigenvalues replacing the sum of the eigenvalues. 

10.20 Definition determinant of an operator, dct T 
Suppose T e C{V). 

• If F = C, then the determinant of T is the product of the eigenvalues 
of T, with each eigenvalue repeated according to its multiplicity. 

• If F = R, then the determinant of T is the product of the eigenvalues 
of 7c , with each eigenvalue repeated according to its multiplicity. 

The determinant of T is denoted by det T. 

If X i , . . . , X m are the distinct eigenvalues of T (or of 7c if V is a real 
vector space) with multiplicities d\ d m , then the definition above implies 

det T = \ d x x ■■■X d ™. 

Or if you prefer to list the eigenvalues with each repeated according to its 
multiplicity, then the eigenvalues could be denoted X\, ... ,X n (where the 
index n equals dim V) and the definition above implies 

det T = X i • • ■ X n . 


10.B 


10.21 Example Suppose T e £(C * * 3 ) is the operator whose matrix is 

3 -1 -2 

3 2-3 

1 2 0 

Then the eigenvalues of T are 1, 2 + 3 i, and 2 — 3/, each with multiplicity 1, 
as you can verify. Computing the product of the eigenvalues, we find that 
det T — 1 • (2 + 3/) ■ (2 — 3/); in other words, det T = 13. 
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The determinant has a close connection with the characteristic polynomial. 
Suppose Ai, . . . , X n are the eigenvalues of T (or of Tq if V is a real vector 
space) with each eigenvalue repeated according to its multiplicity. Then the 
expression for the characteristic polynomial of T given by 10.11 gives the 
following result. 

10.22 Determinant and characteristic polynomial 

Suppose T e C(V). Let n = dim V. Then det T equals (—1)" times the 

constant term of the characteristic polynomial of T. 

Combining the result above and 10.12, we have the following result. 

10.23 Characteristic polynomial, trace, and determinant 

Suppose T € C{V). Then the characteristic polynomial of T can be 

written as 

z n - (trace 7’)z ,! “ 1 + • • • + (— l)"(det T ). 

We turn now to some simple but important properties of determinants. 
Later we will discover how to calculate det T from the matrix of T (with 
respect to an arbitrary basis). 

The crucial result below has an easy proof due to our definition. 

10.24 Invertible is equivalent to nonzero determinant 

An operator on V is invertible if and only if its determinant is nonzero. 

Proof First suppose V is a complex vector space and T e C,(V). The 
operator T is invertible if and only if 0 is not an eigenvalue of T. Clearly this 
happens if and only if the product of the eigenvalues of T is not 0. Thus T is 
invertible if and only if det T yf 0, as desired. 

Now consider the case where V is a real vector space and T e C(V ). 
Again, T is invertible if and only if 0 is not an eigenvalue of T, which happens 
if and only if 0 is not an eigenvalue of 7c (because Tq and T have the same 
real eigenvalues by 9.11). Thus again we see that T is invertible if and only if 
det 7/0. ■ 

Some textbooks take the result below as the definition of the characteristic 
polynomial and then have our definition of the characteristic polynomial as a 
consequence. 
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10.25 Characteristic polynomial of T equals det(z/ - T) 

Suppose T e C{V). Then the characteristic polynomial of T equals 
det(z7 - T). 

Proof First suppose V is a complex vector space. If A, z € C, then A is an 
eigenvalue of T if and only if z — A is an eigenvalue of z7 — T, as can be seen 
from the equation 


—(T - XI) = (z7 — T) — (z — A)7. 

Raising both sides of this equation to the dim V power and then taking null 
spaces of both sides shows that the multiplicity of A as an eigenvalue of T 
equals the multiplicity of z — A as an eigenvalue of z7 — T. 

Let Ai, . . . , A„ denote the eigenvalues of T, repeated according to mul- 
tiplicity. Thus for z e C, the paragraph above shows that the eigenvalues 

of z7 — T are z — Ai z — X n , repeated according to multiplicity. The 

determinant of z7 — T is the product of these eigenvalues. In other words, 

det(z7 — T) = (z — Ai) ■ ■ ■ (z — A„). 

The right side of the equation above is, by definition, the characteristic poly- 
nomial of T, completing the proof when V is a complex vector space. 

Now suppose V is a real vector space. Applying the complex case to Tq 
gives the desired result. ■ 

Determinant of a Matrix 

Our next task is to discover how to compute det T from the matrix of T (with 
respect to an arbitrary basis). Let’s start with the easiest situation. Suppose 
V is a complex vector space, T e C(V), and we choose a basis of V as in 
8.29. With respect to that basis, T has an upper-triangular matrix with the 
diagonal of the matrix containing precisely the eigenvalues of T, each repeated 
according to its multiplicity. Thus det T equals the product of the diagonal 
entries of M{T) with respect to that basis. 

When dealing with the trace in the previous section, we discovered that the 
formula (trace = sum of diagonal entries) that worked for the upper-triangular 
matrix given by 8.29 also worked with respect to an arbitrary basis. Could that 
also work for determinants? In other words, is the determinant of an operator 
equal to the product of the diagonal entries of the matrix of the operator with 
respect to an arbitrary basis? 
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Unfortunately, the determinant is more complicated than the trace. In par- 
ticular, det T need not equal the product of the diagonal entries of M.(T) with 
respect to an arbitrary basis. For example, the operator in Example 10.21 has 
determinant 1 3 but the product of the diagonal entries of its matrix equals 0. 

For each square matrix A, we want to define the determinant of A, denoted 
det A, so that det T — det M(T) regardless of which basis is used to com- 
pute M.(T). We begin our search for the correct definition of the determinant 
of a matrix by calculating the determinants of some special operators. 


10.26 Example Suppose a n e F. Let 


/ 0 
at 


A = 


0 

a 2 0 




V 


@n — 1 0 J 


here all entries of the matrix are 0 except for the upper-right comer and 
along the line just below the diagonal. Suppose rq , . . . , v n is a basis of V and 
T e C{V) is such that M{T, (vi v n )) = A. Find the determinant of T. 


Solution First assume aj ^ 0 for each j — 1, . . . , n — 1. Note that the list 

vi.Fvi, 7 " 2 vi T n ~ l v i equals vi, aiv 2 , a\a 2 V 3 , . . . , ci \ ■ --an-iVn- 

Thus v\,Tv\ T n ~ l v i is lin- 
early independent (because the a’s are 
all nonzero). Hence if p is a monic poly- 
nomial with degree at most n — 1, then 
p(T)v i ^ 0. Thus the minimal poly- 
nomial of T cannot have degree less 
than n . 

As you should verify, T n Vj = a i ■■■a n Vj for each j . Thus we have 
T n = ai ■ ■ ■ a n L Hence z n — a\ ■ ■ ■ a n is the minimal polynomial of T. Be- 
cause n = dim V and the characteristic polynomial is a polynomial multiple 
of the minimal polynomial (9.26), this implies that z n — a \ ■ ■ ■ a n is also the 
characteristic polynomial of T. 

Thus 10.22 implies that 

detT = (— l) n_1 fli • ■ -a n . 

If some a j equals 0, then Tv j = 0 for some j , which implies that 0 is an 
eigenvalue of T and hence det T = 0. In other words, the formula above also 
holds if some aj equals 0. 


Computing the minimal polynomial 
is often an efficient method of find- 
ing the characteristic polynomial, 
as is done in this example. 
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Thus in order to have det T = det M.(T), we will have to make the deter- 
minant of the matrix in Example 10.26 equal to (— l)” _1 <3i • • • a n . However, 
we do not yet have enough evidence to make a reasonable guess about the 
proper definition of the determinant of an arbitrary square matrix. 

To compute the determinants of a more complicated class of operators, we 
introduce the notion of permutation. 

10.27 Definition permutation, perm n 

• A permutation of (1, . . . ,«) is a list (m i, . . . ,m n ) that contains 
each of the numbers 1 , ,n exactly once. 

• The set of all permutations of (1, . . . , n) is denoted perm /;. 


For example, (2, 3, 4, 5, 1) € perm 5. You should think of an element of 
perm n as a rearrangement of the first n integers. 


10.28 Example Suppose a i a n e F and vi, . . . ,v n is a basis of V. 

Consider a permutation (p i p n ) € perm n that can be obtained as fol- 

lows: break (1 into lists of consecutive integers and in each list move 
the first term to the end of that list. For example, taking n — 9, the permutation 

(2,3, 1,5,6,7,4,9,8) 

is obtained from (1, 2, 3), (4, 5, 6, 7), (8, 9) by moving the first term of each of 
these lists to the end, producing (2, 3, 1), (5, 6, 7, 4), (9, 8), and then putting 
these together to form the permutation displayed above. 

Fet T € C{V) be the operator such that 

Tv k = a k v Pk 

for k = 1 Find det T. 


Solution This generalizes Example 10.26, because if (p i p n ) is the 

permutation (2, 3, . . . , n , 1), then our operator T is the same as the operator 
T in Example 10.26. 

With respect to the basis vi , . . . , v n , the matrix of the operator T is a block 
diagonal matrix 


( A1 

0 \ 

\ 0 

A M ) 


where each block is a square matrix of the form of the matrix in 10.26. 
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Correspondingly, we can write V — V\ © • • • © Vm, where each Vj is 
invariant under T and each T\vj is of the form of the operator in 10.26. 
Because det T = (det T\y l ) ■■ ■ (det T\y M ) (because the dimensions of the 
generalized eigenspaces in the Vj add up to dim V), we have 

detr = (—l)" 1 - 1 - - - (— 1 ) nM ~ 1 a 1 ---a n , 

where Vj has dimension n j (and correspondingly each A j has size n j -by -itj) 
and we have used the result from 10.26. 


The number (— 1)” 1_1 ■ ■ • (— I ) nM ~ ] that appears above is called the sign 
of the corresponding peimutation (pi, . . . , p n ), denoted sign( p\ . . ... p n ) 
[this is a temporary definition that we will change to an equivalent definition 
later, when we define the sign of an arbitrary permutation]. 

To put this into a form that does not depend on the particular permutation 
(p i let Aj k denote the entry in row j , column k, of the matrix A 

from Example 10.28. Thus 


A 


j,k ~ 


0 if 7 / Pk\ 
ak if; = Pk- 


Example 10.28 shows that we want 


10.29 det ,4= ^ (sign(mi, A miA ■■■A mn , n ; 

(mi,...,m n )epermn 


note that each summand is 0 except the one corresponding to the permutation 
(pi , . . . , pn) [which is why it does not matter that the sign of the other 
permutations is not yet defined]. 

We can now guess that det A should be defined by 10.29 for an arbitrary 
square matrix A. This will turn out to be correct. We will now dispense with 
the motivation and begin the more formal approach. First we will need to 
define the sign of an arbitrary permutation. 


10.30 Definition sign of a permutation 

• The sign of a permutation (mi, . . . , m n ) is defined to be 1 if the 
number of pairs of integers (j, k) with 1 < j <k < n such that 
j appears after k in the list (mi, . . . , m n ) is even and —1 if the 
number of such pairs is odd. 

• In other words, the sign of a permutation equals 1 if the natural 
order has been changed an even number of times and equals — 1 if 
the natural order has been changed an odd number of times. 


SECTION 10. B Determinant 


313 


10.31 Example sign of permutation 

• The only pair of integers (j, k) with j < k such that j appears after 
k in the list (2, 1, 3, 4) is (1,2). Thus the permutation (2, 1,3,4) has 
sign — 1 . 

• In the permutation (2, 3, . . . , n, 1), the only pairs (j, k) with j < k 
that appear with changed order are (1, 2), (1, 3), . . . , (1, «); because we 
have n — 1 such pairs, the sign of this permutation equals (— l) n_1 (note 
that the same quantity appeared in Example 10.26). 

The next result shows that interchanging two entries of a peimutation 
changes the sign of the permutation. 

10.32 Interchanging two entries in a permutation 

Interchanging two entries in a permutation multiplies the sign of the 
permutation by — 1 . 

Proof Suppose we have two permutations, where the second permutation is 
obtained from the first by interchanging two entries. If the two interchanged 
entries were in their natural order in the first permutation, then they no longer 
are in the second permutation, and vice versa, for a net change (so far) of 1 or 
— 1 (both odd numbers) in the number of pairs not in their natural order. 

Consider each entry between the two Some texts use the term signum 
interchanged entries. If an intermediate w hi c h means the same as sign. 
entry was originally in the natural order 

with respect to both interchanged entries, then it is now in the natural order 
with respect to neither interchanged entry. Similarly, if an intermediate entry 
was originally in the natural order with respect to neither of the interchanged 
entries, then it is now in the natural order with respect to both interchanged 
entries. If an intermediate entry was originally in the natural order with respect 
to exactly one of the interchanged entries, then that is still true. Thus the net 
change for each intermediate entry in the number of pairs not in their natural 
order is 2, —2, or 0 (all even numbers). 

For all the other entries, there is no change in the number of pairs not in 
their natural order. Thus the total net change in the number of pairs not in 
their natural order is an odd number. Thus the sign of the second permutation 
equals — 1 times the sign of the first permutation. ■ 


Some texts use the term signum, 
which means the same as sign. 


Our motivation for the next definition comes from 10.29. 
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10.33 Definition determinant of a matrix, det A 
Suppose A is an n-by-n matrix 

( ^4i,i ■ ■ • Ai,n ^ 

: : 

A n , 1 ■■■ A n , n ) 

The determinant of A, denoted det A, is defined by 

det 4= ^ (sign(mi,...,rn„))A mi>1 ---A mn>n . 

(mi,...,m„)epermn 


10.34 Example determinants 

• If A is the 1 -by- 1 matrix [^4i,i], then det A = A\,\, because perm 1 
has only one element, namely (1), which has sign 1. 

• Clearly perm 2 has only two elements, namely (1,2), which has sign 1, 
and (2, 1), which has sign —1. Thus 

det ( 

To make sure you understand this 
process, you should now find the for- 
mula for the determinant of an arbitrary 
3-by-3 matrix using just the definition 
given above. 


The set perm 3 contains six ele- 
ments. In general, perm n contains 
n ! elements. Note that n ! rapidly 
grows large as n increases. 


10.35 Example Compute the determinant of an upper-triangular matrix 


A = 


( ^i,i 


V 0 


* 


*n,n 


\ 


Solution The permutation (1,2 has sign 1 and thus contributes a term 
of A\ j • • • A nn to the sum defining det A in 10.33. Any other permutation 
(m i, . . . , m n ) € perm /? contains at least one entry mj with nij > j , which 
means that A mj j — 0 (because A is upper triangular). Thus all the other 
terms in the sum in 10.33 make no contribution. 

Hence det A = ■ ■ ■ A n , n . In other words, the determinant of an upper- 

triangular matrix equals the product of the diagonal entries. 
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Suppose V is a complex vector space, T e £(V), and we choose a basis 
of V as in 8.29. With respect to that basis, T has an upper-triangular matrix 
with the diagonal of the matrix containing precisely the eigenvalues of T, 
each repeated according to its multiplicity. Thus Example 10.35 tells us that 
det T — det M.(T), where the matrix is with respect to that basis. 

Our goal is to prove that det T — det M.(T) for every basis of V, not just 
the basis from 8.29. To do this, we will need to develop some properties of 
determinants of matrices. The result below is the first of the properties we 
will need. 

10.36 Interchanging two columns in a matrix 

Suppose A is a square matrix and B is the matrix obtained from A by 

interchanging two columns. Then 

det A = — det B. 

Proof Think of the sum defining det A in 10.33 and the corresponding sum 
defining det B . The same products of Aj^’s appear in both sums, although 
they correspond to different permutations. The permutation corresponding to 
a given product of Ay^’s when computing det B is obtained by interchanging 
two entries in the corresponding permutation when computing det A, thus 
multiplying the sign of the permutation by —1 (see 10.32). Hence we see that 
det A — — det B . m 

If T € C( V) and the matrix of T (with respect to some basis) has two 
equal columns, then T is not injective and hence det T = 0. Although this 
comment makes the next result plausible, it cannot be used in the proof, 
because we do not yet know that det T = det M(T) for every choice of basis. 

10.37 Matrices with two equal columns 

If A is a square matrix that has two equal columns, then det A = 0. 

Proof Suppose A is a square matrix that has two equal columns. Interchang- 
ing the two equal columns of A gives the original matrix A. Thus from 10.36 
(with B = A), we have 

det A — — det A, 


which implies that det A = 0. 


316 


CHAPTER 10 Trace and Determinant 


Recall from 3.44 that if A is an n-by-/t matrix 


^ A\,\ . . . A\, n \ 

^ ~ 1 ]’ 

\ . ■ ■ A n ^ n J 

then we can think of the k th column of A as an w-by- 1 matrix denoted A. y. 


A -,k = 


Some books define the determinant 
to be the function defined on the 
square matrices that is linear as 
a function of each column sepa- 
rately and that satisfies 10.38 and 
det 7 = 1. To prove that such a 
function exists and that it is unique 
takes a nontrivial amount of work. 


( A\ ,k ^ 

V A n,k j 

Note that Ajp, with two subscripts, de- 
notes an entry of A, whereas A.p, with 
a dot as a placeholder and one subscript, 
denotes a column of A. This notation 
allows us to write A in the form 

(A. tl ... A., n ), 


which will be useful. 

The next result shows that a permutation of the columns of a matrix 
changes the determinant by a factor of the sign of the permutation. 


10.38 Permuting the columns of a matrix 

Suppose A = ( A . } i . . . A. ?n ) is an n-by-n matrix and (mi, ... , m n ) 
is a permutation. Then 

det( A, mi ... A., mn ) = (sign(mi,...,m„))detri. 


Proof We can transform the matrix ( A . m , ... A., mn ) into A through a 

series of steps. In each step, we interchange two columns and hence multiply 
the determinant by —1 (see 10.36). The number of steps needed equals the 
number of steps needed to transform the permutation (m\, . . . ,m n ) into the 
permutation (1, by interchanging two entries in each step. The proof 
is completed by noting that the number of such steps is even if (mi , ... , m n ) 
has sign 1, odd if (mi, , m n ) has sign —1 (this follows from 10.32, along 
with the observation that the permutation (!,...,«) has sign 1). ■ 
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The next result about determinants will also be useful. 

10.39 Determinant is a linear function of each column 

Suppose k , n are positive integers with 1 < k < n. Fix n- by-1 matrices 
A.j , . . . , A. } „ except A. k . Then the function that takes an «-by-l column 
vector A.fi to 

det( A. a ... A. ik ... A.,„ ) 

is a linear map from the vector space of n-by-1 matrices with entries in F 
to F. 

Proof The linearity follows easily from 10.33, where each term in the sum 
contains precisely one entry from the k th column of A. ■ 

Now we are ready to prove one of 
the key properties about determinants 
of square matrices. This property will 
enable us to connect the determinant of 
an operator with the determinant of its 
matrix. Note that this proof is considerably more complicated than the proof 
of the corresponding result about the trace (see 10.14). 

10.40 Determinant is multiplicative 

Suppose A and B are square matrices of the same size. Then 

det(A5) = det (BA) = (det A) (det B). 


The result below was first proved 
in 1812 by French mathematicians 
Jacques Binet and Augustin-Louis 
Cauchy. 


Proof Write A = ( Aj 
of A. Also write 


/ B 


B = 


i,t 


\ B n,i 


A. t „ ), where each A. k is an n-by-l column 

B l,n \ 

= ( /?,! ... B,„ ), 

Bn,n / 


where each B. ^ is an //-by- 1 column of B . Let eg denote the /7-by-l matrix 
that equals 1 in the k th row and 0 elsewhere. Note that A eg — A. j, and 
Be k - B. k . Furthermore, B. k = Y,m=\ B m.k^m- 

First we will prove det (AB) — (det ,4 ) (det B). As we observed ear- 
lier (see 3.49), the definition of matrix multiplication easily implies that 
AB — ( AB .,1 . . . AB.^ n ). Thus 
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<\e\.{AB) = det( AB.^\ ... AB.^ n ) 

= det( A(^2 mi = l B mi ,lSmi) ■■■ ^QZm„ = l Bm„,n€m n ) ) 

= det( 5 Z OT j = i B mi ^Ae mi ... X^„ = i Bm„,nAe mn ) 

n n 

— ^ ' ■■■ 'y ' ®mi,i ■ ' ■ B mn ^ n det( Ae m ^ ... Ae mn ), 

m i = l m n = 1 

where the last equality comes from repeated applications of the linearity of det 
as a function of one column at a time (10.39). In the last sum above, all terms 
in which m j = /)?/, for some j ^ k can be ignored, because the determinant 
of a matrix with two equal columns is 0 (by 10.37). Thus instead of summing 
over all m \ .... , m n with each mj taking on values 1 we can sum just 
over the permutations, where the m , ’s have distinct values. In other words, 

det(^4B) = ^ ' B t 77 1 ; i • • • B mn fI det( Ae mi . . . Ae mn ) 

(mi,.. . ,m n )€perm n 

= ^ B m j ; i • • • B mn (sign(m . ,m n j) det A 

(mi,.. . ,m n )€perm n 

= (det 4) ^ (sign(n?i,...,m„))5 OTlj i ■■■B mn> „ 

(m\ ,...,m„)Sperm77 

= (det ^4)(det B), 

where the second equality comes from 10.38. 

In the paragraph above, we proved that det (4 B) = (det .4) (det B). In- 
terchanging the roles of A and B, we have det( BA) = (det 71) (det A). The 
last equation can be rewritten as det( BA) — (det 4) (det B), completing the 
proof. ■ 


Note the similarity of the proof of 
the next result to the proof of the 
analogous result about the trace 
( see 10.15). 


Now we can prove that the determi- 
nant of the matrix of an operator is in- 
dependent of the basis with respect to 
which the matrix is computed. 


io.4i Determinant of matrix of operator does not depend on basis 

Let T € C{V). Suppose u\, u n and v\ , v„ are bases of V. Then 

det .Ad (r, (mi,...,w„)) = de\.M(T, (vi, . . . ,v n j). 
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Proof Let A — A4(I , (: u\ u n ), (vi, . . . , v n )). Then 

det M(T, (mi — , u n )) = det^ _1 (Ad(T, (vi — , v n ))A)^j 

= det((A4(T, (vi,...,v n ))A)A~ * 1 * * * ^ 

= det M.(T, (vi, . . . , v„)), 

where the first equality follows from 10.7 and the second equality follows 
from 10.40. The third equality completes the proof. ■ 

The result below states that the determinant of an operator equals the 
determinant of the matrix of the operator. This theorem does not specify a 
basis because, by the result above, the determinant of the matrix of an operator 
is the same for every choice of basis. 

10.42 Determinant of an operator equals determinant of its matrix 
Suppose T € £(V). Then det T = detA4(T). 

Proof As noted above, 10.41 implies that det M.(T) is independent of which 
basis of V we choose. Thus to show that det T = det M.(T) for every basis 
of V, we need only show that the result holds for some basis of V. 

As we have already discussed, if V is a complex vector space, then choos- 
ing a basis of V as in 8.29 gives the desired result. If V is a real vector space, 
then applying the complex case to the complexification Tq (which is used to 
define det T ) gives the desired result. ■ 

If we know the matrix of an operator on a complex vector space, the result 
above allows us to find the product of all the eigenvalues without finding any 
of the eigenvalues. 

1 0.43 Example Suppose T is the operator on C 5 whose matrix is 

/ 0 0 0 0 -3 \ 

1 0 0 0 6 

0 10 0 0 

0 0 10 0 

v 0 0 0 1 0 / 

No one knows an exact formula for any of the eigenvalues of this operator. 
However, we do know that the product of the eigenvalues equals —3, because 
the determinant of the matrix above equals —3. 
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We can use 10.42 to give easy proofs of some useful properties about 
determinants of operators by shifting to the language of determinants of 
matrices, where certain properties have already been proved or are obvious. 
We carry out this procedure in the next result. 

10.44 Determinant is multiplicative 
Suppose S,T € C{V). Then 

det(ST) = det(7W) = (det S') (det T). 

Proof Choose a basis of V. Then 

det(ST) = det2W(5T) 

= det(Ad(5)Ad(T)) 

= (det A / t(S))(det Af (T)) 

= (det 5) (det T), 

where the first and last equalities come from 10.42 and the third equality 
comes from 10.40. 

In the paragraph above, we proved that det (ST) = (det .S') (det T). Inter- 
changing the roles of S and T , we have det (T. S’) = (det r)(det S). Because 
multiplication of elements of F is commutative, the last equation can be 
rewritten as det (TS) — (det ,S') (det T), completing the proof. ■ 

The Sign of the Determinant 

We proved the basic results of linear algebra before introducing determinants 
in this final chapter. Although determinants have value as a research tool in 
more advanced subjects, they play little role in basic linear algebra (when the 
subject is done right). 

Determinants do have one important 
application in undergraduate mathemat- 
ics, namely, in computing certain vol- 
umes and integrals. In this subsection 
we interpret the meaning of the sign of 
the determinant on a real vector space. Then in the final subsection we will 
use the linear algebra we have learned to make clear the connection between 
determinants and these applications. Thus we will be dealing with a part of 
analysis that uses linear algebra. 


Most applied mathematicians 
agree that determinants should 
rarely be used in serious numeric 
calculations. 
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We will begin with some purely linear algebra results that will also be 
useful when investigating volumes. Our setting will be inner product spaces. 
Recall that an isometry on an inner product space is an operator that preserves 
norms. The next result shows that every isometry has determinant with 
absolute value 1. 

10.45 Isometries have determinant with absolute value 1 

Suppose V is an inner product space and S e C( V) is an isometry. Then 
|detS| = 1. 

Proof First consider the case where V is a complex inner product space. 
Then all the eigenvalues of S have absolute value 1 (see the proof of 7.43). 
Thus the product of the eigenvalues of S, counting multiplicity, has absolute 
value one. In other words, |det S\ = 1, as desired. 

Now suppose V is a real inner product space. We present two different 
proofs in this case. 

Proof 1 : With respect to the inner product on the complexification Vq given 
by Exercise 3 in Section 9.B, it is easy to see that Sq is an isometry on Vq. 
Thus by the complex case that we have already done, we have |det Sc \ — 1 • 
By definition of the determinant on real vector spaces, we have det S — det Sq 
and thus | det 51 = 1, completing the proof. 

Proof 2: By 9.36, there is an orthonormal basis of V with respect to which 
A 4(S) is a block diagonal matrix, where each block on the diagonal is a 
1-by-l matrix containing 1 or —1 or a 2-by-2 matrix of the form 

( cos 9 — sin 9 
sin 9 cos 9 

with 9 € (0, n). Note that the determinant of each 2-by-2 matrix of the form 
above equals 1 (because cos 2 6 + sin 2 9 = 1). Thus the determinant of S, 
which is the product of the determinants of the blocks (see Exercise 6), is the 
product of l’s and — l’s. Hence, |detSj = 1, as desired. ■ 

The Real Spectral Theorem 7.29 states that a self-adjoint operator T on a 
real inner product space has an orthonormal basis consisting of eigenvectors. 
With respect to such a basis, the number of times each eigenvalue appeal's on 
the diagonal of M.(T) is its multiplicity. Thus det T equals the product of its 
eigenvalues, counting multiplicity (of course, this holds for every operator, 
self-adjoint or not, on a complex vector space). 
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Recall that if V is an inner product space and T € C( V), then T* T is a 
positive operator and hence has a unique positive square root, denoted ~JT*T 
(see 7.35 and 7.36). Because ~JT*T is positive, all its eigenvalues are non- 
negative (again, see 7.35), and hence det JT*T > 0. These considerations 
play a role in next example. 


10.46 Example Suppose V is a real inner product space and T e C(V) 
is invertible (and thus det T is either positive or negative). Attach a geometric 
meaning to the sign of det T. 

Solution First we consider an isometry S e C.(V). By 10.45, the determinant 
of S equals 1 or — 1 . Note that 

{v e V : Sv — — v} 

is the eigenspace E(—l,S). Thinking 
geometrically, we could say that this 
is the subspace on which S reverses 
direction. An examination of proof 
2 of 10.45 shows that det S = 1 if 
this subspace has even dimension and 
det S — — 1 if this subspace has odd 
dimension. 

Returning to our arbitrary invertible operator T e C{V), by the Polar 
Decomposition (7.45) there is an isometry S € C(V) such that 

T = SVr*T . 


We are not formally defining the 
phrase “reverses direction” be- 
cause these comments are meant 
only as an intuitive aid to our un- 
derstanding. 


Now 10.44 tells us that 


detr = (det S') (det VY*T). 

The remarks just before this example pointed out that det JT*T > 0. Thus 
whether det T is positive or negative depends on whether det S is positive or 
negative. As we saw in the paragraph above, this depends on whether the 
subspace on which S reverses direction has even or odd dimension. 

Because T is the product of S and an operator that never reverses direction 
(namely, s/T*T), we can reasonably say that whether det T is positive or 
negative depends on whether T reverses vectors an even or an odd number of 
times. 


SECTION 10. B Determinant 


323 


Volume 

The next result will be a key tool in our investigation of volume. Recall that 
our remarks before Example 10.46 pointed out that det J~T*T > 0. 

10.47 | det T | = det y/T*T 

Suppose V is an inner product space and T € C(V). Then 

| det r | = det Vf*T . 


Proof 

By the Polar Decomposition (7.45), Another proof of this result is sug- 

there is an isometry S e C(V) such gested in Exercise 8. 

that 

T = SVT*T. 

Thus 

| det r | = |detS|det Vf*T 
= det VT*T, 

where the first equality follows from 10.44 and the second equality follows 
from 10.45. ■ 

Now we turn to the question of volume in R” . Fix a positive integer n for 
the rest of this subsection. We will consider only the real inner product space 
R" , with its standard inner product. 

We would like to assign to each subset £2 of R" its n -dimensional volume 
(when n — 2, this is usually called area instead of volume). We begin with 
boxes, where we have a good intuitive notion of volume. 

10.48 Definition box 

A box in R" is a set of the form 

{(yi,...,y n ) e R” : xj < yj < xj + r, for j = 1 

where r\, ... ,r n are positive numbers and (xi x n ) 6 R". The num- 

bers ri , . . . , r n are called the side lengths of the box. 

You should verify that when n = 2, a box is a rectangle with sides parallel 
to the coordinate axes, and that when n — 3, a box is a familiar 3-dimensional 
box with sides parallel to the coordinate axes. 
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The next definition fits with our intuitive notion of volume, because we 
define the volume of a box to be the product of the side lengths of the box. 

10.49 Definition volume of a box 

The volume of a box B in R” with side lengths ri , . . . , r n is defined to 

be ri ■ ■ ■ r n and is denoted by volume B. 

To define the volume of an arbitrary 
set £2 c R" , the idea is to write £2 as a 
subset of a union of many small boxes, 
then add up the volumes of these small 
boxes. As we approximate £2 more accurately by unions of small boxes, we 
get a better estimate of volume £2. 

10.50 Definition volume 

Suppose £2 CR". Then the volume of £2, denoted volume £2, is defined 

to be the infimum of 

volume B\ + volume B 2 + ■■■ , 

where the infimum is taken over all sequences B\, B 2 , . . . of boxes in R" 

whose union contains £2 . 

We will work only with an intuitive notion of volume. Our puipose in this 
book is to understand linear algebra, whereas notions of volume belong to 
analysis (although volume is intimately connected with determinants, as we 
will soon see). Thus for the rest of this section we will rely on intuitive notions 
of volume rather than on a rigorous development, although we shall maintain 
our usual rigor in the linear algebra parts of what follows. Everything said 
here about volume will be correct if appropriately interpreted — the intuitive 
approach used here can be converted into appropriate correct definitions, 
correct statements, and correct proofs using the machinery of analysis. 

10.51 Notation T(£ 2 ) 

For T a function defined on a set £2, define r(£2) by 
T (£2) = {Tx:xe £2}. 

For T e F(R" ) and £2 C R", we seek a formula for volume T(£2) in 
terms of T and volume £2. We begin by looking at positive operators. 


Readers familiar with outer mea- 
sure will recognize that concept 
here. 
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10.52 Positive operators change volume by factor of determinant 

Suppose T e £(R” ) is a positive operator and Si C R". Then 

volume T(f2) = (det T)(volume £2). 

Proof To get a feeling for why this result is true, first consider the special 
case where Ai, . . . , X n are positive numbers and T e £(R" ) is defined by 

T(x 1 Xn) = (A 1 * 1 , . . . ,A n x n ). 

This operator stretches the j th standard basis vector by a factor of A j. If B 
is a box in R” with side lengths r\, ... ,r n , then T(B) is a box in R” with 
side lengths Air, . . . , A n r. The box T(B) thus has volume Ai • • • X n r\ ■ ■ ■ r n , 
whereas the box £2 has volume r\-- -r n . Note that det T = A i • ■ ■ A„ . Thus 

volume T( B) — (det T )( volume B) 

for every box B in R". Because the volume of £2 is approximated by sums of 
volumes of boxes, this implies that volume T(£ 2) = (det T)(volume £2). 

Now consider an arbitrary positive operator T € £(R”). By the Real 
Spectral Theorem (7.29), there exist an orthonormal basis e \ , . . . , e n of R” 

and nonnegative numbers Ai, . . . , X n such that Tej = A jej for j = 1 n. 

In the special case where e\, . . . ,e n is the standard basis of R' ? , this operator 
is the same one as defined in the paragraph above. For an arbitrary orthonor- 
mal basis e\, ... ,e n , this operator has the same behavior as the one in the 
paragraph above — it stretches the j th basis vector in an orthonormal basis by 
a factor of A j. Your intuition about volume should convince you that volume 
behaves the same with respect to each orthonormal basis. That intuition, and 
the special case of the paragraph above, should convince you that T multiplies 
volume by a factor of A i • • • X n , which again equals det T. m 

Our next tool is the following result, which states that isometries do not 
change volume. 

10.53 An isometry does not change volume 

Suppose S € £(R”) is an isometry and £2 CR". Then 


volume ,S’(f2) = volume 12 . 
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Proof For x,y e R'\ we have 


Sx - S>|| = \\S(x - ;y)|| 

= \\x-y\\. 


In other words, S does not change the distance between points. That property 
alone may be enough to convince you that S does not change volume. 

However, if you need stronger persuasion, consider the complete descrip- 
tion of isometries on real inner product spaces provided by 9.36. According to 
9.36, S can be decomposed into pieces, each of which is the identity on some 
subspace (which clearly does not change volume) or multiplication by —1 on 
some subspace (which again clearly does not change volume) or a rotation 
on a 2-dimensional subspace (which again does not change volume). Or use 
9.36 in conjunction with Exercise 7 in Section 9.B to write S as a product of 
operators, each of which does not change volume. Either way, you should be 
convinced that S does not change volume. ■ 

Now we can prove that an operator T e £(R”) changes volume by a factor 
of |det T\. Note the huge importance of the Polar Decomposition in the proof. 

10.54 T changes volume by factor of |det T\ 

Suppose T e C( R”) and 11 CR". Then 

volume 7 (£2) = |det T \ (volume £2). 

Proof By the Polar Decomposition (7.45), there is an isometry S € C(V) 
such that 

T = SVY*T. 

If £2 C R", then 7(£ 2) = S(VT*T(a)). Thus 

volume 7(£2) = volume S (v / 7"*r(£2)) 

= volume \fT*T (£2) 

= (det Vr* 7) (volume £2) 

= | det T | (volume £2), 

where the second equality holds because volume is not changed by the isom- 
etry S (by 10.53), the third equality holds by 10.52 (applied to the positive 
operator J~T*T), and the fourth equality holds by 10.47. ■ 
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The result that we just proved leads to the appearance of determinants in 
the formula for change of variables in multivariable integration. To describe 
this, we will again be vague and intuitive. 

Throughout this book, almost all the functions we have encountered have 
been linear. Thus please be aware that the functions / and a in the material 
below are not assumed to be linear. 

The next definition aims at conveying the idea of the integral; it is not 
intended as a rigorous definition. 


10.55 Definition integral, f Q f 

If £2 C R” and / is a real- valued function on £2, then the integral of / 
over £2, denoted f or J Q f (x) dx, is defined by breaking £2 into pieces 
small enough that / is almost constant on each piece. On each piece, 
multiply the (almost constant) value of / by the volume of the piece, then 
add up these numbers for all the pieces, getting an approximation to the 
integral that becomes more accurate as £2 is divided into finer pieces. 

Actually, £2 in the definition above needs to be a reasonable set (for 
example, open or measurable) and / needs to be a reasonable function (for 
example, continuous or measurable), but we will not worry about those 
technicalities. Also, notice that the x in f(x) dx is a dummy variable and 
could be replaced with any other symbol. 

Now we define the notions of differentiable and derivative. Notice that 
in this context, the derivative is an operator, not a number as in one- variable 
calculus. The uniqueness of T in the definition below is left as Exercise 9. 


10.56 Definition differentiable, derivative, a'(x) 

Suppose £2 is an open subset of R" and a is a function from £2 to R" . 
For x e £2, the function a is called differentiable at x if there exists an 
operator T e £(R") such that 


lim 

;y^0 


o(x + y)-a(x) - Ty\\ 

ll.vll 


= 0 . 


If a is differentiable at x, then the unique operator T e £(R' ! ) satisfying 
the equation above is called the derivative of a at x and is denoted by 
a'(x). 


328 


CHAPTER 10 Trace and Determinant 


If n = 1, then the derivative in the 
sense of the definition above is the 
operator on R of multiplication by 
the derivative in the usual sense of 
one-variable calculus. 


The idea of the derivative is that 
for x fixed and ||y || small, 


of I cr(* + y) Ra cr(x) + (o-'(x)) (j): 


because at' (x) e £(R"), this makes 
sense. 


Suppose £2 is an open subset of R" and a is a function from £2 to R". We 
can write 


a(x) - (or(x), . . . ,cr„(x)), 


where each ctj is a function from S2 to R. The partial derivative of a j 
with respect to the k th coordinate is denoted D^Oj . Evaluating this partial 
derivative at a point x e £2 gives D ] ( o } (x). If a is differentiable at x, then the 
matrix of a'(x) with respect to the standard basis of R” contains D ] c a j (x) in 
row j , column k (this is left as an exercise). In other words, 



10.57 


Now we can state the change of variables integration formula. Some 
additional mild hypotheses are needed for / and a' (such as continuity or 
measurability), but we will not worry about them because the proof below is 
really a pseudoproof that is intended to convey the reason the result is true. 

The result below is called a change of variables formula because you can 
think of y = cr(x) as a change of variables, as illustrated by the two examples 
that follow the proof. 

10.58 Change of variables in an integral 

Suppose £2 is an open subset of R” and a : £1 — > R" is differentiable at 
every point of £2. If / is a real- valued function defined on cr(£2), then 


f(y)dy= j f (a(x))|deta / (x)| dx 



Proof Let x e £2 and let T be a small subset of £2 containing x such that / 
is approximately equal to the constant / (ct(x)) on the set a(T). 

Adding a fixed vector [such as cr(x)] to each vector in a set produces 
another set with the same volume. Thus our approximation for a near x using 
the derivative shows that 
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volume ct(F) % volume[(ff / (x))(r)]. 

Using 10.54 applied to the operator o'{x), this becomes 

volume cr ( r ) ss |detcr / (x)|(volume F). 

Let y = er(x). Multiply the left side of the equation above by f(y) and the 
right side by f (a(x)) [because y = ct(x), these two quantities are equal], 
getting 


f(y) volume er(r) w / (a (x))|detcr / (x)| (volume T). 

Now break £2 into many small pieces and add the corresponding versions of 
the equation above, getting the desired result. ■ 

The key point when making a change of variables is that the factor of 
|deta'(x)| must be included when making a substitution y — /(x), as in the 
right side of 10.58. We finish up by illustrating this point with two important 
examples. 


10.59 Example polar coordinates 

Define a : R 2 — > R 2 by 

o(r, 9 ) = (r cos (9, r sin 9), 


where we have used r, 9 as the coordinates instead of xi,X 2 for reasons 
that will be obvious to everyone familiar with polar coordinates (and will 
be a mystery to everyone else). For this choice of a, the matrix of partial 
derivatives corresponding to 10.57 is 

( cos 9 —rsin9 
sin 9 r cos 9 

as you should verify. The determinant of the matrix above equals r, thus 
explaining why a factor of r is needed when computing an integral in polar 
coordinates. 

For example, note the extra factor of r in the following familiar formula 
involving integrating a function / over a disk in R 2 : 




fix, y) dy dx 



f(r cos Q, r sin 9)r dr d9. 
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10.60 Example spherical coordinates 

Define a : R 3 — »• R 3 by 

a(p,(p,9) = (p sin cp cos 9, p sin <p sin 9,p cos<p). 


where we have used p, 9, cp as the coordinates instead of x \ , xi, xj, for reasons 
that will be obvious to everyone familiar with spherical coordinates (and will 
be a mystery to everyone else). For this choice of er, the matrix of partial 
derivatives corresponding to 10.57 is 


( sin cp cos 9 p cos cp cos 9 
sin cp sin 9 p cos <p sin 9 
cos cp — p sirup 


—p sin i p sin 9 
p sin <p cos 6 
0 


as you should verify. The determinant of the matrix above equals p 1 2 simp, 
thus explaining why a factor of p 2 sin <p is needed when computing an integral 
in spherical coordinates. 

For example, note the extra factor of p 2 sin i p in the following familiar 
formula involving integrating a function / over a ball in R 3 : 


/•l /• V l-x 2 r^/l-x 2 -) 

7—1 J— f 1 — X 2 J ~y/ 1 — X 2 — l 


fix, y, z ) dz dy dx 


r r f\ 

Jo Jo Jo 


fip simp cos 9, p sin<p sin 9, p cos <p)p 2 sin<p dp dcp d9. 


EXERCISES 10. B 


1 Suppose V is areal vector space. Suppose T e C{V) has no eigenvalues. 
Prove that det T > 0. 

2 Suppose V is a real vector space with even dimension and T e C( V). 
Suppose det T < 0. Prove that T has at least two distinct eigenvalues. 

3 Suppose T e C{V) and n = dim V > 2. Let Ai, . . . , X n denote the 
eigenvalues of T (or of 7c if V is a real vector space), repeated according 
to multiplicity. 

(a) Find a formula for the coefficient of z"~ 2 in the characteristic 
polynomial of T in terms of A i , . . . , X n . 

(b) Find a formula for the coefficient of z in the characteristic polyno- 
mial of T in terms of A i , . . . , A„ . 
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4 Suppose T € C(V) and c e F. Prove that det(cT) = c diml/ det T. 

5 Prove or give a counterexample: if S.T € C{V), then det (.S' + T) — 
det S + det T. 

6 Suppose A is a block upper-triangular matrix 



\ 


\ 0 A m J 

where each A j along the diagonal is a square matrix. Prove that 
det A = (det^i) ■ ■ ■ (det^ m ). 


7 Suppose A is an n-by-n matrix with real entries. Let S e C(C n ) denote 
the operator on C” whose matrix equals A, and let T e £(R”) denote 
the operator on R" whose matrix equals A. Prove that trace S — trace T 
and det S — det T. 

8 Suppose V is an inner product space and T e C.( V). Prove that 


det T* = detr. 


Use this to prove that |det T\ — det V T*T , giving a different proof than 
was given in 10.47. 

9 Suppose £1 is an open subset of R” and a is a function from ^ to R”. 
Suppose x € £2 and a is differentiable at x. Prove that the operator 
T € £(R”) satisfying the equation in 10.56 is unique. 

[ This exercise shows that the notation o' (x) is justified.] 

10 Suppose T € £(R") and x e R” . Prove that T is differentiable at x and 
T'(x) = T. 

11 Find a suitable hypothesis on a and then prove 10.57. 

12 Let a.b.c be positive numbers. Find the volume of the ellipsoid 



by finding a set £2 C R 3 whose volume you know and an operator 
T e £(R 3 ) such that T{£2) equals the ellipsoid above. 
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A ~\ 296 
Aj , , 76 
Aj, k , 70 
A,*, 76 
A, 109 

C, 2 

deg, 31 
A, 179 
det, 307, 314 
dim, 44 
0,21 
D k , 328 

E(X,T), 155 

F, 4 
F°°, 13 
F m ’”, 73 
F", 6 
F 5 , 14 

G(X,T), 245 

7,52, 296 
•<=rS 207 
Im, 118 
— oo, 31 
/«/. 327 

£(F), 86 
£(F IF), 52 

M(T), 70, 146 
M{v), 84 


7>(F), 30 
n,91 
V m { F),31 
7?(F), 143 
TV, 195 

R, 2 
Re, 118 

a', ?>21 
5, 243 

f, 97 
■v/f, 233 
T\ 103 
F*, 204 

so 
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7c, 277 
T m , 143 
r|[/, 132, 137 
r/77, 137 

7/^, 193 
77°, 104 
(m, v), 166 

F, 16 
||v||, 168 
F', 101 
F/7/,95 
-v, 15 
Fc, 276 
v + 7/, 94 

z, 118 
|z|, 118 


perm, 311 

© Springer International Publishing 2015 
S. Axler, Linear Algebra Done Right, Undergraduate Texts in Mathematics, 
DOI 10.1 007/978-3-3 19-11080-6 


335 


Index 


absolute value, 118 
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in quotient space, 96 
of complex numbers, 2 
of functions, 14 
of linear maps, 55 
of matrices, 72 
of subspaces, 20 
of vectors, 12 
of vectors in F” , 7 
additive inverse 
in C, 3, 4 
in F", 9 

in vector space, 12, 15 
additivity, 52 

adjoint of a linear map, 204 
affine subset, 94 
algebraic multiplicity, 255 
annihilator of a subspace, 104 
Apollonius’s Identity, 179 
associativity, 3, 12, 56 

backward shift, 53, 59, 81, 86, 140 
basis, 39 

of eigenvectors, 157, 218, 
221,224, 268 

of generalized eigenvectors, 
254 

Binet, Jacques, 317 
Blake, William, 203 
block diagonal matrix, 255 


box in R", 323 

Cauchy, Augustin-Louis, 171, 317 
Cauchy-Schwarz Inequality, 172 
Cayley, Arthur, 262 
Cayley-Hamilton Theorem 

on complex vector space, 26 1 
on real vector space, 284 
change of basis, 298 
change of variables in integral, 

328 

characteristic polynomial 

on complex vector space, 26 1 
on real vector space, 283 
characteristic value, 134 
Christina, Queen of Sweden, 1 
closed under addition, 1 8 
closed under scalar multiplication, 
18 

column rank of a matrix, 111 
commutativity, 3, 7, 12, 25, 56, 75, 
79, 144,212 
complex conjugate, 118 
complex number, 2 
Complex Spectral Theorem, 218 
complex vector space, 13 
complexification 

of a vector space, 276 
of an operator, 277 
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conjugate transpose of a matrix, 
207 
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cube root of an operator, 224 
cubic formula, 124 

degree of a polynomial, 3 1 
derivative, 327 
Descartes, Rene, 1 
determinant 

of a matrix, 314 
of an operator, 307 
diagonal matrix, 155 
diagonal of a square matrix, 147 
diagonalizable, 156 
differentiable, 327 
differentiation linear map, 53, 56, 
59,61,62, 69, 72,78, 
144, 190, 248, 294 
dimension, 44 

of a sum of subspaces, 47 
direct sum, 21, 42, 93 
of a subspace and its 

orthogonal complement, 
194 

of null T n and range T n , 243 
distributive property, 3, 12, 16, 56, 
79 

Division Algorithm for 

Polynomials, 121 
division of complex numbers, 4 
dot product, 1 64 
double dual space, 116 
dual 

of a basis, 102 
of a linear map, 103 
of a vector space, 101 

eigenspace, 155 
eigenvalue of an operator, 134 
eigenvector, 134 


Euclid, 275 

Euclidean inner product, 166 

factor of a polynomial, 122 
Fibonacci, 131 
Fibonacci sequence, 161 
field, 10 

finite-dimensional vector space, 30 
Flatland, 6 

Fundamental Theorem of Algebra, 
124 

Fundamental Theorem of Finear 
Maps, 63 

Gauss, Carl Friedrich, 51 
generalized eigenspace, 245 
generalized eigenvector, 245 
geometric multiplicity, 255 
Gram, Jprgen, 182 
Gram-Schmidt Procedure, 1 82 
graph of a linear map, 98 

Halmos, Paul, 27 
Hamilton, William, 262 
harmonic function, 179 
Hermitian, 209 
homogeneity, 52 
homogeneous system of linear 
equations, 65, 90 
Hypatia, 241 

identity map, 52, 56 
identity matrix, 296 
image, 62 
imaginary part, 118 
infinite -dimensional vector space, 
31 

inhomogeneous system of linear 
equations, 66, 90 
injective, 60 
inner product, 166 
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inner product space, 167 
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invariant subspace, 132 
inverse 

of a linear map, 80 
of a matrix, 296 
invertible linear map, 80 
invertible matrix, 296 
isometry, 228, 292, 321 
isomorphic vector spaces, 82 
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Jordan basis, 273 
Jordan Form, 273 
Jordan, Camille, 272 

kernel, 59 

Khayyam, Omar, 117 

Laplacian, 179 
length of list, 5 
Leonardo of Pisa, 131 
linear combination, 28 
Linear Dependence Lemma, 34 
lineal - functional, 101, 187 
linear map, 52 
linear span, 29 
linear subspace, 18 
linear transformation, 52 
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linearly independent, 32 
list, 5 

of vectors, 28 
Lovelace, Ada, 295 

matrix, 70 

multiplication, 75 
of linear map, 70 
of nilpotent operator, 249 
of operator, 146 
of product of linear maps, 75, 
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of r, no 

of T*, 208 
of vector, 84 

minimal polynomial, 263, 279 
minimizing distance, 198 
monic polynomial, 262 
multiplication, see product 
multiplicity of an eigenvalue, 254 

Newton, Isaac, 203 
nilpotent operator, 248, 271 
nonsingular matrix, 296 
norm, 164, 168 
normal operator, 212, 287 
null space, 59 

of powers of an operator, 242 
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of T*, 207 

one-to-one, 60 
onto, 62 
operator, 86 
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projection, 195 
vectors, 169 
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basis, 181 
list, 180 
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permutation, 311 
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polar coordinates, 329 
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positive semidefinite operator, 227 


340 Index 


product 

of complex numbers, 2 
of linear maps, 55 
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range, 61 
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rank of a matrix, 112 
Raphael, 275 
real paid, 118 

Real Spectral Theorem, 22 1 
real vector space, 13 
restriction operator, 137 
Riesz Representation Theorem, 
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Riesz, Frigyes, 187 
row rank of a matrix, 111 

scalar, 4 

scalar multiplication, 10, 12 
in quotient space, 96 
of linear maps, 55 
of matrices, 73 
Schmidt, Erhard, 182 
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Schur’s Theorem, 186 
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self-adjoint operator, 209 
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singular matrix, 296 
Singular Value Decomposition, 
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span, 29 
spans, 30 

Spectral Theorem, 218, 221 
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square root of an operator, 223, 
225, 233, 259 
standard basis, 39 
subspace, 18 
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trace 

of a matrix, 300 
of an operator, 299 
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Triangle Inequality, 173 
tuple, 5 

unitary operator, 229 
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